-- Logs begin at Tue 2022-08-09 10:34:05 UTC, end at Tue 2022-08-09 11:35:49 UTC. -- Aug 09 11:03:54.622758 dv-cinder-852514-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 11:03:54.631116 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 11:03:54 2022] *** Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: nodename: dv-cinder-852514-2-iscsi Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: machine: x86_64 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: clock source: unix Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: pcre jit disabled Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: detected number of CPU cores: 4 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: current working directory: / Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: detected binary path: /usr/bin/uwsgi-core Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: your processes number limit is 47680 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: your memory page size is 4096 bytes Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: detected max file descriptor number: 2048 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: lock engine: pthread robust mutexes Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: thunder lock: enabled Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 11:03:54.636329 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 11:03:54.674418 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: Python main interpreter initialized at 0x558fd64da9c0 Aug 09 11:03:54.674418 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: python threads support enabled Aug 09 11:03:54.674418 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: your server socket listen backlog is limited to 100 connections Aug 09 11:03:54.674418 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: your mercy for graceful operations on workers is 90 seconds Aug 09 11:03:54.674816 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 11:03:54.674816 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: *** Operational MODE: preforking *** Aug 09 11:03:54.675149 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: *** uWSGI is running in multiple interpreter mode *** Aug 09 11:03:54.675149 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: spawned uWSGI master process (pid: 80159) Aug 09 11:03:54.675415 dv-cinder-852514-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 11:03:54.676300 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 1 (pid: 80169, cores: 1) Aug 09 11:03:54.676404 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 2 (pid: 80170, cores: 1) Aug 09 11:03:54.676570 dv-cinder-852514-2-iscsi devstack@keystone.service[80159]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 11:04:00.093871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 11:04:00.093871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Aug 09 11:04:00.094706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 11:04:00.094706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Aug 09 11:04:00.095108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 11:04:00.095108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Aug 09 11:04:00.095207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 11:04:00.095207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Aug 09 11:04:00.142911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 11:04:00.143117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 11:04:00.143195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 11:04:00.143271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 11:04:00.143337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 11:04:00.143488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 11:04:00.143580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 11:04:00.143811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 11:04:00.143933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 11:04:00.144018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 11:04:00.144085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.144898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.145966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.146125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.146213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.146432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.146547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.147900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.148990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.149915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.150939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.151926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.152978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.153853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.154899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.155947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.156154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.156234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 11:04:00.156479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.156560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.156771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.156859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.157840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.158784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.159051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.159145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.159530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.159840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.160932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.161845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.162984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.163982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.164936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.165860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.166998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.167889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.168963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.169982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.170978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.171978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.172917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.173869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.174795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.175941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.176901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.177855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.178914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.179895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.180850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.181961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.182925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.183934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.184907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.185975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.186937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.187924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.188989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.189925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.190949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.191931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.192972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.193894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.194961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.195995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.196944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.197855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.198943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.199943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.200871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.201940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.202937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.203967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.204943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.205921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.206900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.207986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.208923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.209940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.210860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.211996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.212961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.213896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.214943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.215976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.216959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.217890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.218998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.219980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.220996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.221955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222486 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.222994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.223995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.224904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.225935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.226945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.227985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.228907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.229926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.230944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.231954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 11:04:00.232758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 11:04:00.232758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 11:04:00.260333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.260588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.260760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.260760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.260970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.260970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.270630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.270947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.298699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.298958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.299103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.299185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.350555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.351289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.400498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.400679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.401592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.401826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.464896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.465014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.465090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.465204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.465290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.465507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.488861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.489639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.500032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.500366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.512847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.513983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.514061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.514122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.514192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.514348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.514525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 11:04:00.531028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.531233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 11:04:00.531476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.531599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.531734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 11:04:00.531834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 11:04:00.531982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 11:04:00.532255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.532988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.533257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.544372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.544651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.544918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.545104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.545190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.545275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.545507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.545783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.552156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.552682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.555185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.555466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.555703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.555829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.556916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.557985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.558106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.558231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.558432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.580834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.581474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.582692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.582985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.583106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.583260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.587514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.587789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.587910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.588040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.588122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.588448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.594529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.594808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.594930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.595226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.599217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.599345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.599464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.599546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.599771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.599915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.600905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.601129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.601254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.617353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.617636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.617732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.617940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.621895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.622560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.625145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.625444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.628794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.628928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.629958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.633132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.634352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.636299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.636596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.636893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.637187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.637465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.637589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.637726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.640653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.640946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.641240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.641530 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.641806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.642090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.654037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.654336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.654605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.654879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.658409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.658722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.658987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.659248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.662093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.663657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.663943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.666052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.666745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.667031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.667331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.667596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.667734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.667872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.667946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.670614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.670897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.671187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.671473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.671746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.679356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.679643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.679936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.680224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.680536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.680835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.681088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.683079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.683360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.683634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.683913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.684194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.684489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.684741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.697627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.700589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.701328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.703515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.704233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.705107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.706625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.706933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.707195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.707319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.708765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.710247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.710540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.710827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.713628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.715299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.717170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.718183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.718481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.718745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.718868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.719004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.721654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.721970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.722225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.722527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.727057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.727350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.727630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.730438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.730725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.731005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.734655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.734941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.735226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.735506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.735780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.736063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.736366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.736643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.737025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.737302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.737672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.737921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.738073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 11:04:00.738217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.738345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.738447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.738624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.738903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.739170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.739448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.739720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.739993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.740386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.740655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.740932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.741201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.741462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.741717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 11:04:00.771552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558fd64da9c0 pid: 80170 (default app) Aug 09 11:04:00.774688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558fd64da9c0 pid: 80169 (default app) Aug 09 11:04:00.775161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c8a54-4eba-4943-8b1d-1fc9e7b40ac3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:00.775447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c8a54-4eba-4943-8b1d-1fc9e7b40ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:00.775722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c8a54-4eba-4943-8b1d-1fc9e7b40ac3 None None] PATH_INFO: `/v3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:00.780056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Tue Aug 9 11:04:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:08.362111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:08.362799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:08.362799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:08.367232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:08.373506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:08.374207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:08.374347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:08.452934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] Registered counter for database keystone Aug 09 11:04:08.478387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 11:04:08.489238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 11:04:08.673134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 11:04:08.673464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 11:04:08.729978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:08.731577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:08.876227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 11:04:09.015583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:09.087045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] Registered counter for database keystone Aug 09 11:04:09.111430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 11:04:09.123180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 11:04:09.248801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a73deb-3326-4ad7-b779-7b6f7ce1442a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:09.254217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:09.254829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:09.255104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:09.255374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:09.263713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:09.314836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 11:04:09.315103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef70d4a-3484-4f51-88fc-e27c4149d23d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:09.325629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Aug 9 11:04:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:04:09.332414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:09.347491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b0bdc-2ffb-4eeb-a009-4ae4ee6e6a53 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:09.353115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:09.354216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:09.354497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:09.354761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] PATH_INFO: `/v3/projects/69344eba9b82497c957a2d6fd891ec79` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:09.356323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] RBAC: Authorizing `identity:get_project(project_id=69344eba9b82497c957a2d6fd891ec79)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:09.405848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 11:04:09.406427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1416b8-0604-4ba9-a53d-8569edb44b29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:09.410856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Tue Aug 9 11:04:09 2022] GET /identity/v3/projects/69344eba9b82497c957a2d6fd891ec79 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:11.370504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb16a4-e1a6-4782-b5d1-eeeab3924d19 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:11.371065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb16a4-e1a6-4782-b5d1-eeeab3924d19 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:11.371065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb16a4-e1a6-4782-b5d1-eeeab3924d19 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:11.375137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:11.381987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:11.382717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:11.382843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:11.410985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:11.412568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:11.422889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 11:04:11.457738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb16a4-e1a6-4782-b5d1-eeeab3924d19 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:11.480727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb16a4-e1a6-4782-b5d1-eeeab3924d19 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:11.489033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:11.489631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:11.489907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:11.490170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:11.498496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:11.500304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132dfd1-353d-419d-b24b-ad93505a6e8b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:11.509197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Aug 9 11:04:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:04:11.517577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:11.528302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db42e4-5e52-4e8d-ad4f-a34a64279c86 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:11.534755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:11.535368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:11.535641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:11.535906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:11.537393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:11.539232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bea55-fcf2-4a22-b8f9-f63dcfa2c72f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:11.540787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Tue Aug 9 11:04:11 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:13.572202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5489e-9917-430a-b8de-30bb6afb6046 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.572765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5489e-9917-430a-b8de-30bb6afb6046 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.572765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5489e-9917-430a-b8de-30bb6afb6046 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.574075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:13.579418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.579840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.579955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.599658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:13.600846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:13.613567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafa5cc-9958-4fc2-b86a-2255659b3199 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.613847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafa5cc-9958-4fc2-b86a-2255659b3199 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.614123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafa5cc-9958-4fc2-b86a-2255659b3199 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.615414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:13.620306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.620587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.620858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.668554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 11:04:13.668991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 11:04:13.686330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:13.709258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:13.714278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:13.715862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:13.734138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:13.815344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:13.817224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18fdc-deb2-4f3d-bf56-142bb7fb5f28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:13.837011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:13.855622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:13.856321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.856571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.856846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.868086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:13.876902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:13.877884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:13.878896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:13.884294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:13.946064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419315d4-a425-47d4-823c-d1b0c644c986 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:13.948319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:13.968513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:13.985357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:13.986242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:13.986519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:13.986749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:13.987379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4dab1-7c85-47b8-994c-38fcb364e227 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.020421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.021020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.021298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.021890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.022559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:14.022856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.024249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.035913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:14.041525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:14.054618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.056509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.157056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:04:14.163328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:14.166639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:14.252591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076e077-ce59-4353-a7f6-ecd43e948eb4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.270485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ecd1-4067-4b0d-8569-ac4482f4a8be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.287271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.287899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.288169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.288456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.289584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.291024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.292595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:14 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:14.298007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:14.303988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.304856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.305139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.305409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.313542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:14.323772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.325254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:04:14.343617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:14.344573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:04:14.345366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:04:14.345366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:04:14.345366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:14.345366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:04:14.350729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:14.399489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f06f-54e8-49a6-8f36-22ff413c93da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.406452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:14.441133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.441682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.441959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.442228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.443438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.444927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.446490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:14.452409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:14.455800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531a70-281f-4c77-ac3a-9ddf88ffef12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.490795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.491392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.491663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.491929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.492596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:14.492893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.494269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0320b-ef38-4f76-bb3c-97695663510f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.511355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:14.537750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:14.559174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c1f7e-badb-4f64-b02d-5566fe43dfc6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:14.595161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:14.595822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:14.596031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:14.596313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:14.597120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.domain.id=default, user.id=9ebf3927bad74de89873a93b84ee2eae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:14.597495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:14.598888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2f2d2-65f1-4887-a537-f856cabbd0b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:14.606671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Aug 9 11:04:14 2022] GET /identity/v3/role_assignments?user.id=9ebf3927bad74de89873a93b84ee2eae&scope.domain.id=default&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:16.892200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f60d-3537-4f37-94ab-94968746e3ab None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:16.893345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f60d-3537-4f37-94ab-94968746e3ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:16.893892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11f60d-3537-4f37-94ab-94968746e3ab None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:16.895434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:16.901154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:16.901966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:16.902418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:16.929315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:16.930871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:16.978648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74af65-2856-4c5b-892c-de410e17a847 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:16.979248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74af65-2856-4c5b-892c-de410e17a847 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:16.979701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74af65-2856-4c5b-892c-de410e17a847 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:16.981246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/23] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:16.992709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:16.993279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:16.993766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.034502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/24] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:17.035799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:17.037238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:17.069031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:17.191782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:17.214206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:17.248663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:17.288639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:17.290430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3ddd-fe90-4787-a749-cf78b70f39d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:17.358536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:17.359175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.359442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.359700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.377874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4d531f8f-502b-4ef4-b360-2679aff44bb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:17.382617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:17.383620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:17.384444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/26] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:17.388288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4773-ba59-4158-b94a-92cabfd70833 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.388288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4773-ba59-4158-b94a-92cabfd70833 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.388288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4773-ba59-4158-b94a-92cabfd70833 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.388288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/27] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:17.390650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f3da0-f6d5-4081-bf0f-c41bf8399f1a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:17.394043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.394532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.394963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.417503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:17.418957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:17.466038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:17.466576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.466847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.467155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.467775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:17.477589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1993ab8-b1d0-41c9-8130-7d82bc0c6c34 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:17.491346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/28] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:17.492483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:17.558613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:17.600020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef6c62-de25-47e6-af02-e8f115467ef3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:17.637914 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:17.638972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.639412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.639852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.640730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:17.641193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:17.642840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:17.653118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/29] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:17.654234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:17.765096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b1bcb-c997-4a45-9ea2-d55e402fce83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:17.775012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/30] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 9 11:04:17 2022] POST /identity/v3/roles => generated 219 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:04:17.776776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c5e9-1c96-4dd0-a39d-be98720b7715 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.777229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c5e9-1c96-4dd0-a39d-be98720b7715 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.777668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176c5e9-1c96-4dd0-a39d-be98720b7715 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.779102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/31] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:17.784751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c5e9-1c96-4dd0-a39d-be98720b7715 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:17.803596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:17.804671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.805143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.805595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:17.806468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:17.808117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f898e3-94a1-44a5-a174-cb991076d4d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:17.817291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:17.973950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/32] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 9 11:04:17 2022] POST /identity/v3/roles => generated 225 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:17.976293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:17.976772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:17.977480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.000480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:18.002521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:18.087791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/33] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:18.096083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176c5e9-1c96-4dd0-a39d-be98720b7715 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.107862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.117926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.175875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.176472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.176806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.177083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.179923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.181693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.183308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/34] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:17 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:18.191862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.252743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff645344-cabe-49b5-8ecd-6fca294857ee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.271313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.293688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.294326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.294599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.294863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.306109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.307872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:04:18.324370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:18.325255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:04:18.326265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:04:18.326265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:04:18.326265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:18.326265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/35] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:04:18.326700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdb65-6d8f-42e2-b613-df47794ad821 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.332420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.362619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.362839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.363049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.363319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.364561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.365981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b83d-3e0c-45c1-a9ec-869f83ce5109 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.370560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/36] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:18.370560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a05d9-2544-4f51-a634-2730caa9ec91 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.370560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a05d9-2544-4f51-a634-2730caa9ec91 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.370560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a05d9-2544-4f51-a634-2730caa9ec91 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.370560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/37] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:18.371609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4957d-9293-4b04-83fc-75cc0a51904a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.372010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4957d-9293-4b04-83fc-75cc0a51904a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.372227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4957d-9293-4b04-83fc-75cc0a51904a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.373461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/38] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:18.374178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4957d-9293-4b04-83fc-75cc0a51904a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.418653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.442798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab136c32-528b-4d9f-8446-3f50535ed661 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.475282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.475888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.476166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.476465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.476660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4957d-9293-4b04-83fc-75cc0a51904a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.477108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:18.477408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.478804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f8bef-f868-4a6e-b7e0-3161a266125e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.495796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/39] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:18.497153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.497434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.497708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.511850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.512533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.512809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.513080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] PATH_INFO: `/v3/domains/default/users/9ebf3927bad74de89873a93b84ee2eae/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.514961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9ebf3927bad74de89873a93b84ee2eae, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.515575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:18.516427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5833d2-3265-4ace-bb18-1cffde2081c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.516879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:18.532201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/40] 10.222.0.27 () {58 vars in 1361 bytes} [Tue Aug 9 11:04:18 2022] PUT /identity/v3/domains/default/users/9ebf3927bad74de89873a93b84ee2eae/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:18.533412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.533776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.534017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.543096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.562763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:18.564420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:18.602897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/41] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:18.604088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.656107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/42] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:18.656846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.695538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.731739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c59e6a-417b-4b16-82c7-c7578320354e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.770652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb8761-01af-4d85-9eec-290abcced7fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.770948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.772033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.772638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.773052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.774600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.776232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.810688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/43] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:04:18 2022] POST /identity/v3/projects => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:18.810934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.811770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.811878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.812325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.812734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.814140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.816153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:18.818429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/44] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:18.825766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:18.827088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.954104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9835f0e-380a-4dd5-89bb-36ad1c41b6ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.956089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:18.960482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f83b2c-034c-4ce4-9164-4f7f32666e42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:18.996326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:18.996326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:18.996326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:18.996326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:18.997062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:18.997382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef0f9b-9b47-45ec-89cc-9a3174bdca4d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:19.006295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:19.007152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:19.007462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:19.007814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:19.008534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:19.010175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9df8-b0db-44d1-81f0-515a745eeec9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:19.014659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/45] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 9 11:04:18 2022] POST /identity/v3/roles => generated 223 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:19.047618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/46] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:04:18 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:20.530002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d7c85-eb0b-433c-a808-235e6de9284a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:20.530574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d7c85-eb0b-433c-a808-235e6de9284a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:20.530574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d7c85-eb0b-433c-a808-235e6de9284a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:20.532016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/47] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:20.537625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:20.538041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:20.538148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:20.557345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:20.558530 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:20.629261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/48] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:20.652496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d7c85-eb0b-433c-a808-235e6de9284a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:20.728750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:20.762531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d7c85-eb0b-433c-a808-235e6de9284a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:20.796238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:20.796803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:20.797072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:20.797337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:20.808001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:20.812675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:20.813540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:20.814363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/49] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:20.820052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:20.906718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:20.928406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc788c3-88f5-4203-953b-7ddaed0f2e7b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:20.964359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:20.964971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:20.965246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:20.965508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:20.966174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:20.966471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:20.967856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6282f-338d-4a2b-8d45-86d2dd388176 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:20.980026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/50] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:20.985717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:21.089770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d50cc-5f96-44ef-9abe-9de1b56001ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:21.098834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c9cf-01af-4676-9d7a-07e2189538c3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.099118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c9cf-01af-4676-9d7a-07e2189538c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.099387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c9cf-01af-4676-9d7a-07e2189538c3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.100766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/51] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:21.108710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.109160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.109606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.126829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:21.131513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:21.132779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:21.134604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:21.135257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.135497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.136312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.136991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:21.138425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:21.139962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/52] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:20 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:21.145774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:21.202475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/53] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:21.225512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:21.256672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf02f6-5b12-46ed-b5b7-5959a22dc42e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:21.296043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:21.296699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.296974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.297241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.298382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:21.300168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:21.301860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/54] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:21.307484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:21.308598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:21.335436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8c5ae-dc6a-4cc0-b265-fe9206bb7865 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:21.370585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:21.371124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.371395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.371659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.372779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:21.374208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:21.375735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/55] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:21.377177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:21.381844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:21.414173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617074e8-83fc-4ea2-9b72-a527e24d8ca1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:21.447800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:21.448447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.448723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.448993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.449710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.domain.id=default, user.id=9ebf3927bad74de89873a93b84ee2eae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:21.450030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:21.451475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9dd64f-8813-4c3f-ab08-98f96a1608bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:21.457981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/56] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Aug 9 11:04:21 2022] GET /identity/v3/role_assignments?user.id=9ebf3927bad74de89873a93b84ee2eae&scope.domain.id=default&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:21.478834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbea539-ca4a-46fe-ab7b-36c23e0ec30d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:21.514965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:21.515593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:21.515870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:21.516142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:21.516851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:21.518344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675703-e71d-4b63-b382-17c080ded728 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:21.534835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:21.548764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/57] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Aug 9 11:04:21 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:04:23.536672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001cbf4-32d4-45fb-9866-541bb838317f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:23.537219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001cbf4-32d4-45fb-9866-541bb838317f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:23.537219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001cbf4-32d4-45fb-9866-541bb838317f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:23.538558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/58] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:23.547535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:23.548245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:23.548460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:23.576868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:23.578616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:23.656952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/59] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:23.681020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001cbf4-32d4-45fb-9866-541bb838317f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:23.790708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001cbf4-32d4-45fb-9866-541bb838317f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:23.819134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:23.825452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:23.826060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:23.826332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:23.826595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:23.827770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:23.829270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d446c-896b-4d5a-8226-998018df36d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:23.830884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/60] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:23.836809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:23.936942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6493b1-5c1d-4f56-9645-4a89689bb8c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:23.962306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:23.983335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:23.983945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:23.984216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:23.984852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:23.985500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:23.986960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb4297-9ac3-4dd5-a1e7-c780c71a7d65 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:24.017695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/61] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:23 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:27.099975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7996-87cb-426e-a8ae-a868e1f0aa2a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.101007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7996-87cb-426e-a8ae-a868e1f0aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.101509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df7996-87cb-426e-a8ae-a868e1f0aa2a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.103037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/62] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:27.108502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5bf5-236a-4ce7-95ea-e733d58a65c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.109282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5bf5-236a-4ce7-95ea-e733d58a65c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.109755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5bf5-236a-4ce7-95ea-e733d58a65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.129485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b4e5bf5-236a-4ce7-95ea-e733d58a65c0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:27.130976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b4e5bf5-236a-4ce7-95ea-e733d58a65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:27.218908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/63] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:27.255474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df7996-87cb-426e-a8ae-a868e1f0aa2a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.293474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183596-a598-4549-a5de-f83d28d9d54e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.294107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183596-a598-4549-a5de-f83d28d9d54e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.294568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de183596-a598-4549-a5de-f83d28d9d54e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.296076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/64] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:27.301743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f953-e51e-4960-bd40-214354fa330f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.302320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f953-e51e-4960-bd40-214354fa330f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.302800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f953-e51e-4960-bd40-214354fa330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.339634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e904f953-e51e-4960-bd40-214354fa330f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:27.341179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e904f953-e51e-4960-bd40-214354fa330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:27.405354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df7996-87cb-426e-a8ae-a868e1f0aa2a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.455884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.457037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.457594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.458043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.459088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/65] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:27.460692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe249a4-415e-4fef-8e28-47c75f0ec443 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.461132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe249a4-415e-4fef-8e28-47c75f0ec443 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.461524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe249a4-415e-4fef-8e28-47c75f0ec443 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.463137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/66] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:27.468728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.469221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.469650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.487867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:27.493260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:27.494105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/67] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:27.495540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.498851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:27.501260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:27.545485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:27.563415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:27.591536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/68] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:27.592635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.621641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a581-1e78-4649-95ae-9b70369308aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.668341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.669670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.670219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.670590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.683369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:27.688797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:27.689849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/69] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:27.690854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.719669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a581da7-ec81-47bd-86d0-953e48e97686 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.755317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.755921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.756195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.756508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.757172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:27.757471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:27.758849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb547b84-ea82-4ae5-9537-89ace3fddfc4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:27.767419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/70] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:27.768818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee578-9149-4532-9e59-e271f2ff7da3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.769225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee578-9149-4532-9e59-e271f2ff7da3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.769500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee578-9149-4532-9e59-e271f2ff7da3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.770822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/71] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:27.774453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee578-9149-4532-9e59-e271f2ff7da3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.808075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:27.808250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156be730-90b3-40d7-93bd-568659ae66ca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.825094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:27.842198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.842849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.843124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.843389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.854119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:27.858732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:27.859607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/72] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:27.860675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:27.883462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee578-9149-4532-9e59-e271f2ff7da3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.919790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.920425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.920703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.920970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.921646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:27.921943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:27.923306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac0847-93aa-460e-a88c-04e07b3c2ee1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:27.932857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/73] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:27.934067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.934346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.934950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.952085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:27.953276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:27.960927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f59f-2a27-4867-b547-54c96a14d991 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:27.993881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:27.994430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:27.994711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:27.994980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:27.996170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:27.997645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efd6ff-c619-41b8-b088-a623b3e2b901 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:27.999218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/74] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:27 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.000417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9e7fd-13cb-44fe-89c5-0745c1b7c4b7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.000697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9e7fd-13cb-44fe-89c5-0745c1b7c4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.000966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9e7fd-13cb-44fe-89c5-0745c1b7c4b7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.002110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/75] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:28.002867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9e7fd-13cb-44fe-89c5-0745c1b7c4b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.030467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/76] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:28.031312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.056548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.068672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.113864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9e7fd-13cb-44fe-89c5-0745c1b7c4b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.144196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085bde-1064-44c8-a145-08752ca6d317 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.146796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.147408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.147685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.147952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.148644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:28.148945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.150336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.160066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/77] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.160890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.178444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.179075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.179349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.179644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.194475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.195907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0873cce-36d9-40d5-bad0-0baf7d0bad91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.197575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/78] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.198690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.198964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.199229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.217016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:28.218223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:28.260732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7159-977b-47be-a045-475e1b9d53fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.270422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.283166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/79] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:28.284258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.297397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.298016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.298285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.298536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.298913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.369553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.371032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.381400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737a71d-3702-4779-a13d-affa3c59d1ac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.415377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.415995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.416288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.416559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.417692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.419129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.420949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/80] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.421408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.468612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/81] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:28.472977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.482854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.500531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.519518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf94a1-ff5f-45ba-8836-1b7e25271bf5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.552499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.553124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.553404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.553678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.554824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.556297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.557877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/82] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.559752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.571070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b0a93-85d4-4507-ac54-e6a9f187bc70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.603841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.604541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.604812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.605071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.659696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50358dc2-e717-4833-a069-5f9fec5189bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.670335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.671807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.681569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.695046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.695673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.695954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.696228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.707395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:28.713629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:28.714505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:28.715517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/83] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:28.716354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.727291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.773665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/84] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:28.774429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.816579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c05f8-5f09-4e23-b54d-d2a39a6cc83e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.851307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.851921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.852192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.852472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.853138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.854585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.873595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99579409-9935-4bd4-b4f6-19f4079343fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:28.876384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/85] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:28 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:28.877284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.886071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:28.918588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:28.921648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:28.921648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:28.921648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:28.922456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:28.925611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:28.932991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/86] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:28.932991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:28.962217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:29.007623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5d63f-7e66-4ea8-8cb9-051a5de93736 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.044861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.045399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.045673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.046887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.071001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb428-c89e-43ec-bcf0-13f484338d85 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.117959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.119333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.119617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.119893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.120882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:29.121191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.123100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.132830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/87] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.133675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.139421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.140967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.192669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:29.214917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:29.258910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/88] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:29.259947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.267655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac6495-6d19-4d0e-8a53-d0016bebdcf4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.314751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.316305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.317306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.317757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.319593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.321521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.323206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/89] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.324960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.370568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a86c20-235c-4f00-806e-70117e3ae5e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.451616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.451616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.451616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.451616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.452010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=9ebf3927bad74de89873a93b84ee2eae) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:29.452010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.453509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.455087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be786923-b947-4d01-8258-7f20a1767f89 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.462903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:29.479486 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/90] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/role_assignments?user.id=9ebf3927bad74de89873a93b84ee2eae&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.480177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.516921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:29.600770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.602574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.603139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.603601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.605125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.607679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.613900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/91] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.615766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.717407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663cbe1-7c86-47dc-bd51-46439e1df92c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.769134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.769848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.770131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.770430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.772187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.773823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.775485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/92] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.781891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:29.783021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e254cc-3925-4602-b9b5-277ba817f2e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:29.839808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:29.840455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:29.840733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:29.841004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:29.844741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:29.844741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:29.844741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:29.857867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/93] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:29.866900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:30.022961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:30.075541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:30.084344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa63bc-d670-47b4-9240-d483d219d2ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:30.136127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c101-83bc-4319-a16c-942522072b57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:30.189042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:30.189722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:30.189875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:30.190042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:30.190301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:30.190396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:30.190640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:30.190901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:30.191675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:30.191974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:30.193525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:30.199936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/94] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:30.329330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:30.330925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:30.478771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:30.487131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/95] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:30.493004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:30.634892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3947359-a9c6-46cc-b844-c2aa62e8619f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:30.636435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:30.678916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:30.679549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:30.679826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:30.680096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] PATH_INFO: `/v3/projects/0302a012c18e412bb8652cc8242e3f26` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:30.684966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] RBAC: Authorizing `identity:get_project(project_id=0302a012c18e412bb8652cc8242e3f26)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:30.686441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a00c8-117e-409d-bff7-e2ade8aba7fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:30.688104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/96] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:30 2022] GET /identity/v3/projects/0302a012c18e412bb8652cc8242e3f26 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:30.694340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:30.858885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45cd58-49af-41da-b99f-df73d3bdf728 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:30.900418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:30.901029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:30.901306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:30.901580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:30.902302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=0302a012c18e412bb8652cc8242e3f26, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:30.902601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:30.904036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6dfcb-3fe2-4882-a694-df301e8287e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:30.918304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/97] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:30 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=0302a012c18e412bb8652cc8242e3f26&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:31.748485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b02eb-cc77-42a4-bcd0-e5afa2e2d3ba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:31.748994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b02eb-cc77-42a4-bcd0-e5afa2e2d3ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:31.749098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b02eb-cc77-42a4-bcd0-e5afa2e2d3ba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:31.750351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/98] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:31.755912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeceb3-d413-4612-aab6-92af66c58d2e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:31.756366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeceb3-d413-4612-aab6-92af66c58d2e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:31.756582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeceb3-d413-4612-aab6-92af66c58d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:31.785564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bceeceb3-d413-4612-aab6-92af66c58d2e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:31.786794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bceeceb3-d413-4612-aab6-92af66c58d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:31.859444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:31.871574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/99] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:31.906438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b02eb-cc77-42a4-bcd0-e5afa2e2d3ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.000917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070eb8-9742-4b0c-a64c-c5b97bfe28c0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.001402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070eb8-9742-4b0c-a64c-c5b97bfe28c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.001805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070eb8-9742-4b0c-a64c-c5b97bfe28c0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.003912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/100] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:32.009666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc56ef-71c7-4201-b708-a04ff7d74c7e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.010283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc56ef-71c7-4201-b708-a04ff7d74c7e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.010833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc56ef-71c7-4201-b708-a04ff7d74c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.035666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1fbc56ef-71c7-4201-b708-a04ff7d74c7e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:32.037365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1fbc56ef-71c7-4201-b708-a04ff7d74c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:32.073748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b02eb-cc77-42a4-bcd0-e5afa2e2d3ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.138357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.139625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.140083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.140546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.157955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:32.165862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:32.166677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:32.167462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:32.174218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/101] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:32.175819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78dbc-6eee-4de2-b709-1e0a14b4254b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.176308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78dbc-6eee-4de2-b709-1e0a14b4254b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.176784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78dbc-6eee-4de2-b709-1e0a14b4254b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.177707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/102] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:32.179422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/103] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:32.184497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.184715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.187636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.188107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.220315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:32.222907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:32.244656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:32.316634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb552ea-42c1-4f7e-98b2-7067fca610d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.352041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.353196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.353701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.354148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/104] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:32.354325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.355090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.355389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:32.355867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:32.357559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:32.366663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/105] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:32.373889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.457066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:32.523014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a5506-4065-4b5e-8304-469ad34f6338 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.554932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:32.570617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc201cb-f073-4e8b-bacf-635c91a63eb1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.572017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.572908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.574900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.575369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.576690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:32.579974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:32.586805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/106] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:32.587806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.625236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.626229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.626692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.627350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] PATH_INFO: `/v3/users/4b7bcabbca8b4b0bb1789bde0134d633` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.641662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] RBAC: Authorizing `identity:get_user(user_id=4b7bcabbca8b4b0bb1789bde0134d633)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:32.643842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:32.645764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/107] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/users/4b7bcabbca8b4b0bb1789bde0134d633 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:32.647020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.769216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664448a-8ebe-4957-9c99-3f8127dd2b47 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.804635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:32.809915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505343a-6220-43b2-92bb-e3fb4e33269e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:32.811375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.812286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.812804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.813286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.824390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:32.829428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:32.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:32.831258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:32.833523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/108] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:32.839275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.858168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:32.859350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.859859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.860666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.861524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:32.863757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056f833-395f-4c2d-a281-71233f942dd7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:32.899462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/109] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:04:32 2022] POST /identity/v3/projects => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:32.901874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203cf2-6211-44ee-9346-7f9f0e839061 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:32.902560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203cf2-6211-44ee-9346-7f9f0e839061 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:32.903014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de203cf2-6211-44ee-9346-7f9f0e839061 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:32.905284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/110] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:32.910997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203cf2-6211-44ee-9346-7f9f0e839061 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:32.920295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:32.995764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd917a-187a-488c-b1bb-3bccd2304f7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.038184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.038984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.039427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.039795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.098142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.126365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.127949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8581-df0d-43ce-bef8-58e61f60e7ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.133528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de203cf2-6211-44ee-9346-7f9f0e839061 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.178159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.179163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.179685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.180166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.181225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:33.181749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.183446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621b49-4443-4b71-9581-05dcbf42c2c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.192979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/111] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:33.194514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.194989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.195473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.214436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:33.216105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:33.255889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/112] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:33.261623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815721-49cb-40f1-8bf2-b8cfad1f02c5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.261623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815721-49cb-40f1-8bf2-b8cfad1f02c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.261623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815721-49cb-40f1-8bf2-b8cfad1f02c5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.261623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/113] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:33.261623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0815721-49cb-40f1-8bf2-b8cfad1f02c5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.291987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.302190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/114] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:33.303401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.352420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.376400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0815721-49cb-40f1-8bf2-b8cfad1f02c5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.416294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.416294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.416294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.416294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.416686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.417225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed24c28-a834-4852-92fb-e40ad9af1499 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.417323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a509d8-f09c-411b-8b4d-8efdcfafa7bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.418902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/115] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:33.420240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.420543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.420821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.446623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:33.448491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:33.458432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.459110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.459392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.459720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.460907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.462463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.464522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/116] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:33.465496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.544477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/117] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:33.545457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.558052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.608201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd5a4-543c-4601-9c4b-557d3a91613b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.652338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.653318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.653816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.654275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.658413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.668742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:33.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:33.675922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:33.676712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:33.678931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/118] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:33.680112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.691786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19b482-af1c-4604-9164-2abf0877ebcd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.738956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.739599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.739877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.740152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.741337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.742766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.744416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/119] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:33.745183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.806555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d077075-500d-48b9-a948-3e21e350562a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.830867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.844661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.845268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.845545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.846050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.846807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=4b7bcabbca8b4b0bb1789bde0134d633) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:33.847115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:33.848630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:33.861569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/120] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/role_assignments?user.id=4b7bcabbca8b4b0bb1789bde0134d633&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:33.861569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.861889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1455d2-4563-4bed-bd35-77eabab677af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:33.898516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:33.899262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.899601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.899936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.916976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-630ead59-771d-4ee4-b966-317a69a1a09c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:33.924848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:33.925913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:33.926837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:33.928117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/121] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:33.929240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05035e-f0b6-45a4-950d-9bf0c5e4242c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:33.929589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05035e-f0b6-45a4-950d-9bf0c5e4242c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:33.930453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05035e-f0b6-45a4-950d-9bf0c5e4242c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:33.931196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/122] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:33.932030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05035e-f0b6-45a4-950d-9bf0c5e4242c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:33.937051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:33.992850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf48f1-0576-4f9a-974f-d181b85abf62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.034299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.035418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.036018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.036574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.037618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:34.038110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.039878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.051848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/123] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.053199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.065484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05035e-f0b6-45a4-950d-9bf0c5e4242c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.103224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.103914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.104192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.104492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/9ebf3927bad74de89873a93b84ee2eae/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.106459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, user_id=9ebf3927bad74de89873a93b84ee2eae, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.108027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b49c6-e3f9-48d5-845c-f9e31a95a221 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.116322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.128364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/124] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:33 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/9ebf3927bad74de89873a93b84ee2eae/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:34.130219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.130742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.131205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.152328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:34.154088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:34.227707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49916d05-06bc-4c1d-a60d-a85e882bacee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.251845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.270629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/125] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:34.276873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.311797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.312854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.313970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.314985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.315854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:34.318647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.320293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.333128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/126] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.334456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.431305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0982e-d4f8-4c80-aba1-e113f668a61d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.446558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853f600-2ef5-4d8d-b60c-9ba0e4092711 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.459700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.473848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.474487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.474857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.475160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.476462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.478350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.480206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/127] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.481061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.494338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.495659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.496363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.497013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.511897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:34.520171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:34.521702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:34.522446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:34.524741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/128] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:34.525984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.551382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.602564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702bc95-8669-407b-813a-5d487e946618 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.643425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906d45e-23f1-434f-b3fb-37497b50490d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.647912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.648577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.648856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.649129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.650273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.651735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.653793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/129] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.654633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.696404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.697384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.697858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.698329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.699664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.701375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.703403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/130] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.709526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.771648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bfc3c-d65c-4d6b-9875-62034c67888e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.782916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.808389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:34.820312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:34.821149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:34.822286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:34.822727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:34.823753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:34.825363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:34.826890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:34.846067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/131] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:34.847259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:34.958316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547878-884d-4580-9ecd-695f6bc85cad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:34.968106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc629bf7-9343-4980-b747-ac3078290675 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.005490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.006844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.007262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.007666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/b554c50e7a13439e8f3d08f498235e9d/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.010682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, user_id=b554c50e7a13439e8f3d08f498235e9d, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.012290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.038120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/132] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:34 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/b554c50e7a13439e8f3d08f498235e9d/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:35.039419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.042455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.043589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.044753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.045394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.051948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.053807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.055723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/133] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:34 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:35.061766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.150457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:35.240896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb09f9-32a6-4cb5-885b-3bc7e4361ab3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.274079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34c94-01b4-4e24-8cf1-19e2eb88b42d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.291782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:35.310779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.311424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.311703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.311974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.316775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.318238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.320069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/134] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:35 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:35.325350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.336747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.337410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.337690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.337960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/b554c50e7a13439e8f3d08f498235e9d/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.340456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, user_id=b554c50e7a13439e8f3d08f498235e9d, role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.341988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259a36e-f7e1-495c-bbaf-76b6ef3b7b42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.359075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/135] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:35 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/b554c50e7a13439e8f3d08f498235e9d/roles/7f08073ba42c48ff8060c007983602f6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:04:35.360632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dde23-06eb-456e-89e5-42e7418c5098 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.360919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dde23-06eb-456e-89e5-42e7418c5098 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.361239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dde23-06eb-456e-89e5-42e7418c5098 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.363052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/136] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:35.372152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48971740-1b0d-421b-943b-07ada126f611 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.372580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48971740-1b0d-421b-943b-07ada126f611 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.372921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48971740-1b0d-421b-943b-07ada126f611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.405583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-48971740-1b0d-421b-943b-07ada126f611 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:35.406832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48971740-1b0d-421b-943b-07ada126f611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:35.456635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2581b4-1e83-4b23-ae3e-376787dd73e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.482303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:35.499109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.499662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.499939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.500205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] PATH_INFO: `/v3/projects/0302a012c18e412bb8652cc8242e3f26` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.501404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] RBAC: Authorizing `identity:get_project(project_id=0302a012c18e412bb8652cc8242e3f26)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.503194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.504944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/137] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:35 2022] GET /identity/v3/projects/0302a012c18e412bb8652cc8242e3f26 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:35.511721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.522964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/138] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:35.561650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48971740-1b0d-421b-943b-07ada126f611 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.618295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f37ec-8f65-43c4-b0c3-e036755df039 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.655652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.656425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.656705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.656979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] PATH_INFO: `/v3/projects/0302a012c18e412bb8652cc8242e3f26/users/b554c50e7a13439e8f3d08f498235e9d/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.659011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] RBAC: Authorizing `identity:create_grant(project_id=0302a012c18e412bb8652cc8242e3f26, user_id=b554c50e7a13439e8f3d08f498235e9d, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.660492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.675487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/139] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:35 2022] PUT /identity/v3/projects/0302a012c18e412bb8652cc8242e3f26/users/b554c50e7a13439e8f3d08f498235e9d/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:35.731513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:35.738723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48971740-1b0d-421b-943b-07ada126f611 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.790526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.791178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.791456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.791724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.797080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.798582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4feae-f394-49bf-8b65-d1b3922168cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:35.800665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/140] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:35.806298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:35.886608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:35.954734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53601-395b-41f6-a03d-235ce9b90cbd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:35.991143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:35.991763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:35.992015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:35.992299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:35.992966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:35.994476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe905b58-b1e1-45fb-9d15-15f2e0fa624b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:36.021662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/141] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:35 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:36.506421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad7a6-550b-4249-98ce-16e8e01b27ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:36.507494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad7a6-550b-4249-98ce-16e8e01b27ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:36.507978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad7a6-550b-4249-98ce-16e8e01b27ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:36.509543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/142] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:36.523014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d42df-989e-465d-9629-f0086757605e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:36.523437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d42df-989e-465d-9629-f0086757605e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:36.523850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d42df-989e-465d-9629-f0086757605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:36.557668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf0d42df-989e-465d-9629-f0086757605e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:36.559423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf0d42df-989e-465d-9629-f0086757605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:36.635218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/143] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:36.659787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad7a6-550b-4249-98ce-16e8e01b27ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:36.788174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705273ac-0469-432c-9a50-e54c46597e60 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:36.788494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705273ac-0469-432c-9a50-e54c46597e60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:36.788737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705273ac-0469-432c-9a50-e54c46597e60 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:36.790080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/144] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:36.795137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:36.795414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:36.795685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:36.824064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ad7a6-550b-4249-98ce-16e8e01b27ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:36.826867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:36.829275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:36.883574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:36.894077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:36.894724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:36.894999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:36.895268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:36.929004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:36.935469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:36.936416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:36.937211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:36.939520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/145] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:36.945908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:36.964252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/146] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:37.002279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:37.011038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.082537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086f5df-0f56-4428-806d-c298726f763d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.122610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.123610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.124088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.124589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.125466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:37.127046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.128681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.146603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/147] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:37.154378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.201046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afa3f9-b0fa-4656-98ff-95dcbc171d0e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.276233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.277294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.277774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.278257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.304739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:37.312060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:37.312966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:37.313727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:37.324470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/148] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:37.326805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:37.329309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.344683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39066986-b7cb-41c0-bc55-3d87d98a316f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.370581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:37.415988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.416691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.416965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.417234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] PATH_INFO: `/v3/users/4b7bcabbca8b4b0bb1789bde0134d633` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.419767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] RBAC: Authorizing `identity:get_user(user_id=4b7bcabbca8b4b0bb1789bde0134d633)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.421317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.423088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/149] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/users/4b7bcabbca8b4b0bb1789bde0134d633 => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:37.429072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.481805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f9d15-9d29-4d7b-950e-c5659429bb1b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.521681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.522302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.522582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.522851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.523518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:37.523816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.525232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.535765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/150] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:37.542614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.611013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d54fa-86d3-426e-a543-ea19cf928534 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.674975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.675294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.675552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.675817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.676982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.678870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.680790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/151] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:37.687402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.721893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:37.731457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:37.752054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54226a1-fed0-4930-80c0-62166c0f5a01 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.837042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.837697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.837977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.838247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] PATH_INFO: `/v3/users/9ebf3927bad74de89873a93b84ee2eae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.845046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] RBAC: Authorizing `identity:get_user(user_id=9ebf3927bad74de89873a93b84ee2eae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.846891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.848619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/152] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/users/9ebf3927bad74de89873a93b84ee2eae => generated 247 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:37.856296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:37.864452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38aeaa-de47-400a-b91d-9b6f8e7b267e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:37.951306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:37.952048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:37.952362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:37.952723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/4b7bcabbca8b4b0bb1789bde0134d633/roles/74e9aa4660764bb0b022e7bcbc90ef86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:37.957898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, user_id=4b7bcabbca8b4b0bb1789bde0134d633, role_id=74e9aa4660764bb0b022e7bcbc90ef86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:37.959569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:37.986546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/153] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:37 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/users/4b7bcabbca8b4b0bb1789bde0134d633/roles/74e9aa4660764bb0b022e7bcbc90ef86 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:04:38.067968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaacbe8-0cd1-48f8-8b9e-b61ce5116bdc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:38.115781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:38.116352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.116634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.116907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:38.162364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:38.223135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:38.224642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:38.327146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/154] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:04:38.332908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:38.471684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:38.496027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89c7d7-8163-4377-a379-012114b1c7bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:38.538174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:38.539155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.539639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.540167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:38.544938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:38.546613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c358e-c52f-4430-ae51-49c235993977 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:38.548499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/155] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:38 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:38.554571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:38.602636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c97b0-0ece-4886-b5d6-42ca25c5de94 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.603255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c97b0-0ece-4886-b5d6-42ca25c5de94 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.603732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c97b0-0ece-4886-b5d6-42ca25c5de94 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:38.605302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/156] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:38.610851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.611428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.612344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:38.639412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:38.649308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:38.650812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:38.696739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad47171-659a-4bf5-a459-07e18e7318e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:38.757027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:38.757649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.757940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.758196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:38.758937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=9ebf3927bad74de89873a93b84ee2eae) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:38.759266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:38.760727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:38.767338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/157] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:38 2022] GET /identity/v3/role_assignments?user.id=9ebf3927bad74de89873a93b84ee2eae&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:38.795744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/158] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:38.831829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:38.943988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c498b-a74d-4fec-942d-6fe0aac4aac2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:38.970971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:38.989285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:38.989921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:38.990199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:38.990471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.006055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-72ef452e-81f1-447d-a9a8-3ed2ca8b7a29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:39.011393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:39.012391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:39.013812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/159] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:39.018937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.079406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9b87-712d-42a0-8f5f-3a575f0283a9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.079919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9b87-712d-42a0-8f5f-3a575f0283a9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.079919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9b87-712d-42a0-8f5f-3a575f0283a9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.081805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/160] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:39.103373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.103699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.103925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.133659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:39.135883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:39.172755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:39.206025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd45a0d-5156-4b7b-87a1-2bcc5daad50d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.267083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/161] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:39.273767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.274436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.274699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.274967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.275688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:39.276005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:39.277432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:39.287569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/162] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:39.293202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.320961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.470832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b38c-9c0e-47d5-a732-5a247ec68dbd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.501094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a068-43fc-4030-8a57-04f9471fbb84 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.504624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:39.539659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.540206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.540507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.540774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.542004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.542654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.542926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.543191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.547423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:39.549087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:39.551126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/163] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:39.553532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:39.556243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:39.559095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:39.559938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:39.560727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/164] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:04:39.567243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.570130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:39.682965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f21f-06d0-4add-ba3e-d0cf57a2a9ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.717970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.718570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.718841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.719105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.719768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:39.720061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:39.721523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:39.729999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/165] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:39.735698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.766722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74251a7-1b58-4018-9e77-6b8ecb02840f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.800908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:39.843820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b4fcf-f613-4f3a-8455-aadf88a1e542 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:39.849706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.850461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.850826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.851184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.881164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:39.882042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:39.882452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:39.882927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:39.884396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:39.886134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:39.892425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/166] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:39.894269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:39.967091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:40.015090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8611432-1479-469d-9e5a-03a4e7fb385d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:40.025886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.027274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.047597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:40.054688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:40.055553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.056007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.056477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.132904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.135053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.242509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/167] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:04:40.248619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:40.259040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/168] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:40.264610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:40.283896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:40.357489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc455b7a-2821-44dc-bc15-17d174f17aa8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:40.395384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:40.397261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.399814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.400282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.401650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.403297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4c33-2cef-4890-9199-033d728e8c9b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.406488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/169] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:40 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:40.413193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe925b2-0f42-4239-be1e-47aacdf0abbd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.414313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe925b2-0f42-4239-be1e-47aacdf0abbd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.414904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe925b2-0f42-4239-be1e-47aacdf0abbd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.417444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/170] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:40.423093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961e8e0-7769-4d41-9792-ea4fe37dc172 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.423824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961e8e0-7769-4d41-9792-ea4fe37dc172 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.424998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961e8e0-7769-4d41-9792-ea4fe37dc172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.455104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:40.462564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1961e8e0-7769-4d41-9792-ea4fe37dc172 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:40.464064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1961e8e0-7769-4d41-9792-ea4fe37dc172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:40.500676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f7fc4-e97b-4807-8c1e-b8eca1c6a79c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:40.550115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:40.551208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.551698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.552153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.553706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.557096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.558947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/171] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:40 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:40.562215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:40.597487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/172] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:40.600404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fa8-d6fe-4b87-b6d0-820ed673efdc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.601792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fa8-d6fe-4b87-b6d0-820ed673efdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.603898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13fa8-d6fe-4b87-b6d0-820ed673efdc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.606753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/173] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:40.607948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13fa8-d6fe-4b87-b6d0-820ed673efdc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:40.674794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:40.766761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b96c7-daa5-48c5-ae00-2cd0d4d8ddc5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:40.808166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13fa8-d6fe-4b87-b6d0-820ed673efdc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:40.832205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:40.833570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.834057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.834527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.835441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:40.836424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.838313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd8e57-8e6a-4a1e-8637-8909b606971f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.846282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/174] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:40 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:40.849274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698b03-eb66-44ae-88f6-874838a5cd1b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.849731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698b03-eb66-44ae-88f6-874838a5cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.855088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e698b03-eb66-44ae-88f6-874838a5cd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.855267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:40.855267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:40.855267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:40.855267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:40.855540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:40.855540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:40.856422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:40.868458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/175] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:40 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:40.869600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:40.894885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8e698b03-eb66-44ae-88f6-874838a5cd1b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:40.897797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e698b03-eb66-44ae-88f6-874838a5cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:40.903514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.056171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/176] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:41.061892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfea4d8-760e-4bc5-9d21-c17d4e0ee8df None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.062178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfea4d8-760e-4bc5-9d21-c17d4e0ee8df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.062447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfea4d8-760e-4bc5-9d21-c17d4e0ee8df None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.064032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/177] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:41.081083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.088388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092b57d-ad27-4137-8d52-01b446c8f0dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.092463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.092742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.093017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.114044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:41.115255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:41.149078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.149664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.149946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.150220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.173843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:41.179269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:41.180137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:41.180961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:41.185855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/178] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:41.186743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.191809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/179] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:41.197440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.293719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.308088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc78d9b-9c8d-48f6-af41-ec4fa059c05b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.323070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc4a3b-f6ba-4b3f-9a8e-54e97cf90f71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.358039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.359593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.360040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.360737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.361692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:41.362343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.364611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.368421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.369151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.369498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.369836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.375608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/180] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:41.380350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.388858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:41.397487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:41.398379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:41.399175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:41.401716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/181] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:41.402640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.424451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.487682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893038a-6b11-4cfe-ac70-0c54bb7b13e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.516211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67df29-793c-4380-b3d6-fba39cf18209 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.517389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.525244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.525878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.526154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.526419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.537813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:41.542955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:41.543824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/182] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:41.545432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.557732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.558386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.558656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.558918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.574653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.576607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb804828-c488-4dfc-81c7-b3bc97f829a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/183] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8724d72-a7dd-404b-8558-abe7e6f5d5f9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8724d72-a7dd-404b-8558-abe7e6f5d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8724d72-a7dd-404b-8558-abe7e6f5d5f9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/184] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:41.583984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8724d72-a7dd-404b-8558-abe7e6f5d5f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.655971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ce8a-fb47-4aff-a489-e6ee12cc290a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.668133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.692936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.693544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.693818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.694085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.694746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:41.695046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.696515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.701083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8724d72-a7dd-404b-8558-abe7e6f5d5f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.709675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/185] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:41.709675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.739659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.740280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.740560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.741055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.741469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:41.741760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.743134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c35f-2dee-41de-bcaf-6fb5d87d0d7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.751322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/186] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:41.752530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.752805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.753068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.769460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.772345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:41.773505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:41.815976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734d692-d13e-43f4-a285-c6093072fc6b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.837366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/187] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:41.838188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.861712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.862616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.862896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.863166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.906937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:41.933958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.935431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb11cd-49e8-414f-94a7-05eadd407a79 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.944869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2137514-f9df-4516-81bf-6038eba011cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:41.979962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:41.980623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:41.980895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:41.981162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] PATH_INFO: `/v3/users/b554c50e7a13439e8f3d08f498235e9d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:41.982284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] RBAC: Authorizing `identity:get_user(user_id=b554c50e7a13439e8f3d08f498235e9d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:41.983734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:41.985342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/188] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/users/b554c50e7a13439e8f3d08f498235e9d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:41.986146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:41.987494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.035434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/189] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:42.036734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2a2d6-3851-4eaa-8cfb-aa6215008afb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.037006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2a2d6-3851-4eaa-8cfb-aa6215008afb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.037267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2a2d6-3851-4eaa-8cfb-aa6215008afb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.038477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/190] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:42.039148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2a2d6-3851-4eaa-8cfb-aa6215008afb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.085212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590ddcc-bbad-4ea9-943c-a53845d1001e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.113685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.119022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.119719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.120100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.120399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.121548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.122995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.124733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/191] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:41 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:42.125393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.145651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2a2d6-3851-4eaa-8cfb-aa6215008afb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.183410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.183941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.184199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.184500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.185714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.187150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.188778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/192] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:42.189555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.211415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.240684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bb274-a493-478f-bd63-9b2a22d1d2ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.275327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.275947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.276218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.276508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.291262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb80a-fd0c-4bb3-a4d6-b647f0c0ce3a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.326460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.327077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.327344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.327626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.328964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.330443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784d4d-8c5e-4cd8-841b-f111a24dec64 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.332238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/193] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:42.333210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.333479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.333749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.349940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.350914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:42.352096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:42.352364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.354130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.417610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/194] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:42.418440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.439061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.466807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/195] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:42.467628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.520370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bafef-d8ee-4fbd-825b-c771ccb82fa8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.557774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.558365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.558634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.558894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.565338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.568242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6df9c-031b-461c-a9fd-23fb7d6524c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.602169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.602761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.603031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.603282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.603916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.605407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.628584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.630045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.631347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/196] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:42 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:42.632167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.663846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.747125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6349ebf-6d28-4b37-9704-ab03022694c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.754707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/197] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:42.759948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.790863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.791472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.791742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.792008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.792768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=2450cc65622148368084296549d12328) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:42.793067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:42.794508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:42.800899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/198] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/role_assignments?user.id=2450cc65622148368084296549d12328&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:42.800899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:42.832879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.891588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3a4f6-3b0b-4f0b-a861-1bd7613b03dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.911910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:42.941770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:42.941770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:42.942068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:42.942494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:42.953407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06406416-0130-43c7-a854-b7119ee53537 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:42.955712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:42.961542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:42.962479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:42.963254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/199] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:42.963887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.004743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.005394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.005670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.005939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] PATH_INFO: `/v3/projects/0302a012c18e412bb8652cc8242e3f26` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.007090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] RBAC: Authorizing `identity:get_project(project_id=0302a012c18e412bb8652cc8242e3f26)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.008611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.010406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/200] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/projects/0302a012c18e412bb8652cc8242e3f26 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.011015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.104360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963fe6-4926-4234-8dd4-91bbdfb4dae0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.112808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:43.125067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2604-72f4-4367-bc75-091867e681b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.145869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.147280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.147738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.148295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.149626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.151375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.153306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/201] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:42 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.154254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.168349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.168781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.169055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.169322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.170051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:43.170283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.171671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.180669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/202] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.181598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.208254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:43.267546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1250e-9ea1-4f57-8e0a-b90b39f749d5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.285093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b1261-edbf-491b-850c-b833c8114ef4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.305857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.306798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.307259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.307716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.308836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=0302a012c18e412bb8652cc8242e3f26, user.id=b554c50e7a13439e8f3d08f498235e9d) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:43.309394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.311095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.318259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/203] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/role_assignments?user.id=b554c50e7a13439e8f3d08f498235e9d&scope.project.id=0302a012c18e412bb8652cc8242e3f26&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.319489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.319886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.320556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.320830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.321100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.321842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=2450cc65622148368084296549d12328) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:43.322141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.323745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.330173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/204] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/role_assignments?user.id=2450cc65622148368084296549d12328&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.442026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47987a19-f3d3-46e7-a67a-adc61a80b886 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.455469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:43.485336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.486616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.487250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.487716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] PATH_INFO: `/v3/users/4b7bcabbca8b4b0bb1789bde0134d633` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.489273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] RBAC: Authorizing `identity:get_user(user_id=4b7bcabbca8b4b0bb1789bde0134d633)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.491031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.493208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/205] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/users/4b7bcabbca8b4b0bb1789bde0134d633 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:43.498521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.522480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:43.640109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dc247-6155-49c7-b1b4-96697a1acf65 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:43.675916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:43.677063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:43.677526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:43.678023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:43.750642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:43.753024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:43.768993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:43.857443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/206] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:43.864278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:43.980860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5e21f-85a7-490a-bce6-eda706f9dc89 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:44.019014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:44.019654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:44.019927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:44.020195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:44.021684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:44.023130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687202de-cea7-4e21-b22c-dcd1c8e81e17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:44.024950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/207] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:43 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:44.031027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:44.162688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad23272-9d3d-4533-a5ae-786be7a588af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:44.177463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:44.202769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:44.203372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:44.203642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:44.203956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:44.204711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, scope.project.id=13abcee6071741f590ffaf32659ac1c6, user.id=4b7bcabbca8b4b0bb1789bde0134d633) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:44.205630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:44.207102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e671927-f0bc-4398-b5fc-5d6ab1418455 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:44.213318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/208] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:44 2022] GET /identity/v3/role_assignments?user.id=4b7bcabbca8b4b0bb1789bde0134d633&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:45.321601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8323-f6bd-4de5-8b5d-e38d50fde24e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.322161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8323-f6bd-4de5-8b5d-e38d50fde24e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.322161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a8323-f6bd-4de5-8b5d-e38d50fde24e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.323789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/209] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:45.329706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f75690-c713-491e-a82f-1cdd9cd76f3f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.330818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f75690-c713-491e-a82f-1cdd9cd76f3f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.331344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f75690-c713-491e-a82f-1cdd9cd76f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.350599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0b69-95a5-4537-bae0-6fa6b155b34f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.350932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0b69-95a5-4537-bae0-6fa6b155b34f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.351134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee0b69-95a5-4537-bae0-6fa6b155b34f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.351634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c5f75690-c713-491e-a82f-1cdd9cd76f3f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:45.352489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/210] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:45.353272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5f75690-c713-491e-a82f-1cdd9cd76f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:45.358244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.358940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.359216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.382639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:45.383838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:45.430094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/211] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:45.436509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbebef-7cd2-4b33-bd2a-5be3b57804e4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.436509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbebef-7cd2-4b33-bd2a-5be3b57804e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.436509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbebef-7cd2-4b33-bd2a-5be3b57804e4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.436509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/212] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:45.438327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:45.438976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.439532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.439969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.459969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:45.461176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:45.466759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/213] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:45.469748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.536532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/214] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:45.538216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.587753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcee7e5-45c1-4172-821e-133a4bd79f68 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:45.594389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:45.625057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:45.625805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.626126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.626480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.639379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:45.644482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:45.645421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:45.646212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/215] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:45.647175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.674291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3c1f2-f53d-414c-9331-5823e64dc9f4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:45.695425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:45.713555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:45.714186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.714456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.715035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.728051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:45.733211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:45.734102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:45.734996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/216] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:45.737448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.777958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd9602-68ef-4130-b596-612e4b71263a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:45.814282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:45.815002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.815298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.815568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.816725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:45.818165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:45.820131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/217] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:45.820921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.855851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:45.860203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b836ab-9dc6-4d99-864d-604ee2607a7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:45.899036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:45.899644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.899915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.900178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.900874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:45.901173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:45.902604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8bb91-e1dc-4183-b836-a64f42bf6d71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:45.911659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/218] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:45.913038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc613b-d5e0-4c86-813d-069f35591ec4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.913322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc613b-d5e0-4c86-813d-069f35591ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.913988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc613b-d5e0-4c86-813d-069f35591ec4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.914833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/219] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:45.914833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc613b-d5e0-4c86-813d-069f35591ec4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:45.955546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5b093-ba0e-449f-8f40-e2f4bf0c378c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:45.957344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:45.993011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:45.993606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:45.993874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:45.994139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:45.994791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:45.995082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:45.996465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.004998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/220] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:46.005808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.035076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc613b-d5e0-4c86-813d-069f35591ec4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.069154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.069765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.070057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.070308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.070958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.072452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d27b-df92-4816-860a-5fe37a37a287 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.108383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/221] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:45 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:46.108383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.108383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.108383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.116573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016283e4-f637-4a8e-b89c-e12c45efcfe0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.127873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:46.129507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:46.141926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.157477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.158321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.158595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.158963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.160165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.161806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.169051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/222] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:46.169051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.208477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.213777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/223] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:46.217234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.290255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2229fb-9616-47ed-bddc-827547f0ac88 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.329096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.329993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.330159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.330295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] PATH_INFO: `/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.346701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] RBAC: Authorizing `identity:get_user(user_id=e91c6f0aa9a54b1987f1ac9eba4fee3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.347549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ca30d-8f75-47ae-903e-422006db49fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.348199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.351748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/224] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:46.353380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.391979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.392622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.392894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.393157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.395230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.397084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.398928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/225] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:46.401154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.418694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.469401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.482436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64277fb6-a906-4089-8db1-13bd184f6558 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.524701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd2ed-5b29-40ae-9aed-7b440b70c9e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.526678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.527321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.527598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.528825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.541629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:46.548024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:46.549190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/226] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:46.550176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.576734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.577591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.578006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.578945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.662610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.664006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.677366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fad96-9d66-4560-8d27-691183a43d9f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.680621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.717146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.718092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.718092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.718344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/2450cc65622148368084296549d12328/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.720229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, user_id=2450cc65622148368084296549d12328, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.722026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:46.740056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/227] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:46 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/2450cc65622148368084296549d12328/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:46.740696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.758070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.792814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/228] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:46.799383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:46.946411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677cedf-932d-43bd-8a3d-c350de4b8c1a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.955333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4653-dd38-427a-b65e-94e4297018ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:46.985304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:46.987042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:46.987775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:46.988179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:46.988579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:46.989303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:46.989653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:46.991036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.004671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/229] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:47.006810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:47.012711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:47.013551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:47.013967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:47.014399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:47.015750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:47.017422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.019266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/230] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:46 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:47.027156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:47.092427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:47.121612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060008c-0f83-4d7a-8101-7e49c9ffe6ff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:47.148952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78320b42-e420-4444-ac43-65e520af19f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:47.159648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:47.160485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:47.160629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:47.160838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:47.161976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:47.163556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.165647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/231] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:47 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:47.171057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:47.187389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:47.188376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:47.189084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:47.190071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:47.191504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=e91c6f0aa9a54b1987f1ac9eba4fee3e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:47.192238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:47.193934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.199659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/232] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:47 2022] GET /identity/v3/role_assignments?user.id=e91c6f0aa9a54b1987f1ac9eba4fee3e&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:47.285072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7d3b2-15ef-443d-8527-c36fba9e3a0c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:47.322651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:47.323285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:47.323561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:47.323832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:47.325004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:47.326463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02715e0c-823b-467a-90bd-d25095959a0b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.328377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/233] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:47 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:47.334268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:47.391862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:47.460472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df157797-e18e-4466-8c1e-0bc08d0db28d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:47.512970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:47.513988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:47.514462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:47.514945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/2450cc65622148368084296549d12328/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:47.517385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, user_id=2450cc65622148368084296549d12328, role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:47.519058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeaee0-150e-4ca2-82de-7e54ae5e6b86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:47.546347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/234] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:47 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/2450cc65622148368084296549d12328/roles/7f08073ba42c48ff8060c007983602f6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:48.522559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f628-cbba-407e-9e5d-eb37434bc09b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:48.523085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f628-cbba-407e-9e5d-eb37434bc09b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:48.523085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20f628-cbba-407e-9e5d-eb37434bc09b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:48.524445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/235] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:48.529952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:48.530793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:48.531293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:48.561209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:48.562708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:48.649578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/236] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:48.685668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20f628-cbba-407e-9e5d-eb37434bc09b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:48.706629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:48.802000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20f628-cbba-407e-9e5d-eb37434bc09b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:48.836895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:48.837564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:48.837844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:48.838113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:48.849101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7bd423f8-9cdd-43e5-b285-50e6404bc4a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:48.854226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:48.855098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:48.855896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/237] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:48.862285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:48.904929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:48.942123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516b-6746-49a9-bfe6-b73a92c43946 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:48.942459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516b-6746-49a9-bfe6-b73a92c43946 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:48.942658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516b-6746-49a9-bfe6-b73a92c43946 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:48.944291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/238] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:48.965525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:48.965853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:48.966156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:48.987518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:48.988735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:49.058472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0a04f-a58c-4e11-90fc-8d4873703ba1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.060197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:49.066592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/239] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:49.093135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.166801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.167476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.167747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.168011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.168736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:49.169027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:49.170451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:49.198566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/240] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:49.205774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed96a8-3c23-4271-b4a5-836457963b72 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.209266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.243146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.243772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.244047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.244346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.245878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:49.247595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:49.249539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/241] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:49.255125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.329063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:49.366117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8098c-2b08-417c-aaa8-1c4e2977bb19 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.408284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.409080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.409451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.409827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.410752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:49.413136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118d272-46cc-4f4d-a47a-4810e2d5be68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:49.439570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:49.443416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/242] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:49 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:04:49.444043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98017c02-a146-40ad-afa3-90856ee13d08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.449426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c17db-3e49-4221-9655-9ec993227775 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.450029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c17db-3e49-4221-9655-9ec993227775 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.450537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c17db-3e49-4221-9655-9ec993227775 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.457657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/243] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:49.460056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cede-f183-4c80-bc07-ea16e05544b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.460723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cede-f183-4c80-bc07-ea16e05544b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.461190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cede-f183-4c80-bc07-ea16e05544b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.503751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e254cede-f183-4c80-bc07-ea16e05544b3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:49.506526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e254cede-f183-4c80-bc07-ea16e05544b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:49.531049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.532488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.533205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.533767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] PATH_INFO: `/v3/users/00717b5051ce4e5c9c4d38de91b44de9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.562960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] RBAC: Authorizing `identity:get_user(user_id=00717b5051ce4e5c9c4d38de91b44de9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:49.565850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:49.569101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/244] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/users/00717b5051ce4e5c9c4d38de91b44de9 => generated 297 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:49.574548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.627532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/245] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:49.645871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117871-16ce-480a-ae68-7d3b90c0c86f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.646791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117871-16ce-480a-ae68-7d3b90c0c86f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.647377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d117871-16ce-480a-ae68-7d3b90c0c86f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.649193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/246] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:49.655904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.656626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.657092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.681744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:49.683414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:49.729245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258da4e-717d-4136-8b8e-f01ee3b3407a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.746866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:49.773681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.774251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/247] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:49.774356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.774545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.774808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.775897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.808689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:49.858264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:49.859789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:49.887967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afbed-653a-4b5d-97dc-e66d23382c09 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:49.945496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:49.946136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:49.946412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:49.946684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:49.961462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:49.968389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:49.969280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:49.970072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/248] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:49.977019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:49.992830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/249] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:04:49.993960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.032305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.064500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.097002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3107307-cc02-4340-be3b-001ee846387e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.120305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b44d3-94af-4dfa-a099-f3a784fc420a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.139882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.140873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.141435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.141898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.153693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6ca902f6-3702-4fb5-81cf-99f0ea58637d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:50.158911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:50.159931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:50.160737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/250] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:50.162938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.163751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.164181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.164634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.165481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:50.166004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.166524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507ea5-7a3a-48e9-bca6-84094bc4275e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.166875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507ea5-7a3a-48e9-bca6-84094bc4275e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.167211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507ea5-7a3a-48e9-bca6-84094bc4275e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.168154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.168937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/251] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:50.169788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2507ea5-7a3a-48e9-bca6-84094bc4275e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.178821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/252] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.184252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.290211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2507ea5-7a3a-48e9-bca6-84094bc4275e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.292686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.294132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60d7e-9904-4151-b038-fa05febc6c1c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.313636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.330069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.330849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.331237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.331568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.332798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.333042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.333844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.334289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.334388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa84b-201b-431f-9a22-0b6636ccbf9f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.334809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.335640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:50.336105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.336221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/253] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.337430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.337766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.338092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.338304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.349082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/254] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.350151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.361491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:50.362974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:50.434285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/255] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:50.441570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.457378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c81b3c-f5d9-4788-8bb6-9c689bdb298b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.494081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.495248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.495699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.496175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.497516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.499148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.501060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/256] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.502144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.565964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453c46c-7f4e-41b7-b169-470483a76bc0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.567836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.603312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.604151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.604449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.604795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.605551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=00717b5051ce4e5c9c4d38de91b44de9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:50.605933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.607616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.613507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a46-76a3-4508-b155-4398d19c5e62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.613634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/257] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/role_assignments?user.id=00717b5051ce4e5c9c4d38de91b44de9&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.614672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.653439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.654496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.655074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.655574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] PATH_INFO: `/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.657113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] RBAC: Authorizing `identity:get_user(user_id=e91c6f0aa9a54b1987f1ac9eba4fee3e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.659057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.661131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/258] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:50.662193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.732467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f81027-bea4-4a2c-9148-bbfa7f5972e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.775668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df44d-d9c2-4903-9d5e-ff51242961b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:50.783221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.783868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.784137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.784429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.796944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:50.797627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:50.803984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:50.804968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:50.805762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/259] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:50.811506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:50.834932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:50.835979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:50.836502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:50.836995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:50.844037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:50.927256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:50.928766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:50.952473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86c5f-caae-48b5-a254-d6dca17fa657 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.029920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.030654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.030924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.031192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.033945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.035413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.037963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/260] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:51.038849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.046930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/261] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:51.047757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.067381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:51.165313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e583a7d-06f6-4c8d-8b64-338713f41972 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.191197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:51.208501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.209174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.209453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.209726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/e91c6f0aa9a54b1987f1ac9eba4fee3e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.211671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, user_id=e91c6f0aa9a54b1987f1ac9eba4fee3e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.213191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.221005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305831a-0557-493a-a4e7-b46677ce898e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.240321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/262] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:51 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/e91c6f0aa9a54b1987f1ac9eba4fee3e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:51.240941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.281638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.282218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.282768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.283126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.284219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:51.284561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.286200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.306273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/263] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:04:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:51.312489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.471051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:51.490546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bb349-4dbf-48cc-a39f-04236929c6c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.527337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9632f7d-000a-4e63-b01a-58c33a274c64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.572475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.573505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.573978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.574438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] PATH_INFO: `/v3/users/2450cc65622148368084296549d12328` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.575871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] RBAC: Authorizing `identity:get_user(user_id=2450cc65622148368084296549d12328)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.577570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.579379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/264] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:51 2022] GET /identity/v3/users/2450cc65622148368084296549d12328 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:51.585384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.586401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.587053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.587333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.587600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.593030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.594528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.596888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/265] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:51 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:51.606672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:51.635564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:51.779747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2b9f3-b3bb-42a2-89c5-7c821c143c22 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.798038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830dce0-c54b-4c1c-851d-91b2460401db None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:51.844741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.845852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.847043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.847687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.872969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:51.873583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:51.873856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:51.874127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:51.874889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=2450cc65622148368084296549d12328) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:51.875200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.880290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ee630-7168-454a-9ad4-a92f435c521a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:51.883311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/266] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:51 2022] GET /identity/v3/role_assignments?user.id=2450cc65622148368084296549d12328&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:51.971372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:51.973298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.007392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.075987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed305-4a6e-4c42-bf24-81dd8cef7946 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.076360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed305-4a6e-4c42-bf24-81dd8cef7946 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.076564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fed305-4a6e-4c42-bf24-81dd8cef7946 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.078131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/267] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:52.089760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbda7d-fdc2-4115-a9f4-d3a46c1033e7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.090087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbda7d-fdc2-4115-a9f4-d3a46c1033e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.090290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbda7d-fdc2-4115-a9f4-d3a46c1033e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.110425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/268] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:04:52.116550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.125423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87bbda7d-fdc2-4115-a9f4-d3a46c1033e7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:52.128622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87bbda7d-fdc2-4115-a9f4-d3a46c1033e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:52.129110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.270488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/269] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:52.272525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6013-7843-43e2-9409-f4e50ca0e3fa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.273004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6013-7843-43e2-9409-f4e50ca0e3fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.273505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6013-7843-43e2-9409-f4e50ca0e3fa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.274935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/270] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:52.282456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.284558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.285202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.311156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:52.312543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:52.313869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee4a7-2e75-4cee-a2d7-6fb9ffef6e26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.339561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.356110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.357931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.358204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.358510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.360224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:52.362868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.364387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/271] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:52.365637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.399600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/272] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:52.400830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.497690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.517704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee484e-2682-472a-b10c-efd2c2e48f7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.539099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cb19c-5b96-4403-a197-9ae59ac1f0db None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.553186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.554178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.554651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.555113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.565919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:52.570938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:52.571805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:52.572624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:52.575140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.575394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/273] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:52.575724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.576137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.576439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.577004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.577169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=2450cc65622148368084296549d12328) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:52.577466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:52.579161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.586200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/274] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/role_assignments?user.id=2450cc65622148368084296549d12328&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:52.587129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.641219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.692423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1f28-43f1-4b62-9332-e0d3a4ebd5c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.732708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.733784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.734298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.734759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.736186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:52.737937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.741118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/275] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:52.747030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.763182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b14d-45d3-4c05-b33c-8875a77f2ac3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.802421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.807913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.809074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.810639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.810910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.812190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:52.813030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:52.815367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.828101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/276] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:52.834497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:52.895261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa967ed-c231-43fc-a675-00e3616eeb46 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.921052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:52.950661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:52.951273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:52.951549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:52.951811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:52.952635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:52.954116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:52.959106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eef5b-f05b-45b0-a77f-5a7cafbd51a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:52.983838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/277] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:04:52 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:04:53.006330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:53.007452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.007953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.008449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] PATH_INFO: `/v3/users/2b104df131844ab3a98ae75326113aa0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.026841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] RBAC: Authorizing `identity:get_user(user_id=2b104df131844ab3a98ae75326113aa0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:53.028820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d8f8e-d63e-4b1f-b0d9-295d256b6bb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:53.031327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/278] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:52 2022] GET /identity/v3/users/2b104df131844ab3a98ae75326113aa0 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:53.037265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:53.166573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb140b-15ba-433d-a135-fb6e390c2bf3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:53.217368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:53.218338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.218820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.219296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.220541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, user.id=2b104df131844ab3a98ae75326113aa0, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:53.221094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:53.222787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31c969-01f9-4b70-a128-e7bb57efb1e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:53.229027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/279] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 9 11:04:53 2022] GET /identity/v3/role_assignments?user.id=2b104df131844ab3a98ae75326113aa0&scope.system=all&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:53.274427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517dcf4-b6f9-4b58-be47-35bfe6f5ea74 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.274751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517dcf4-b6f9-4b58-be47-35bfe6f5ea74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.274962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517dcf4-b6f9-4b58-be47-35bfe6f5ea74 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.276348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/280] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:53.282272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.282642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.283101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.305187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:53.306732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:53.348094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:53.389345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/281] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:53.412209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517dcf4-b6f9-4b58-be47-35bfe6f5ea74 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:53.473177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:53.534291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517dcf4-b6f9-4b58-be47-35bfe6f5ea74 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:53.581664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:53.582644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.583118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.583553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.599480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:53.605022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:53.605908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:53.606697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:53.612754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/282] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:53.619253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:53.743790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6ebae-81d1-4ee1-870b-97ec07b4a7d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:53.789094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:53.789986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:53.790427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:53.790859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:53.791658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:53.792093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:53.793667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:53.807841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/283] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:53.813717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:53.954982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a37dcb-0c26-427e-930b-25f58a18057b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:53.974514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:54.002949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:54.003592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.003868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.004138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] PATH_INFO: `/v3/users/00717b5051ce4e5c9c4d38de91b44de9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.005326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] RBAC: Authorizing `identity:get_user(user_id=00717b5051ce4e5c9c4d38de91b44de9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:54.006778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:54.008546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/284] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:53 2022] GET /identity/v3/users/00717b5051ce4e5c9c4d38de91b44de9 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:54.014046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:54.054346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:54.137150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c835d54-5b67-4e09-a1da-ab249bb3d24a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:54.186919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:54.187565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.187844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.188163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.189715 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:54.191152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bba12-da71-4ee0-8066-488180627227 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:54.192810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/285] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:54 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:54.198687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:54.276848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5fab-3167-4f1a-ab02-c3aa87d82a23 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.277130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5fab-3167-4f1a-ab02-c3aa87d82a23 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.277311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5fab-3167-4f1a-ab02-c3aa87d82a23 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.278894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/286] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:54.284789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.285070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.285347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.312843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:54.314119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:54.342632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e435-7a4a-4219-afec-f78d6c2d1919 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:54.410629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:54.411344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.411625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.411895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/00717b5051ce4e5c9c4d38de91b44de9/roles/74e9aa4660764bb0b022e7bcbc90ef86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.413929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, user_id=00717b5051ce4e5c9c4d38de91b44de9, role_id=74e9aa4660764bb0b022e7bcbc90ef86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:54.417627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:54.442114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/287] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:54.444571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/288] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:04:54 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/users/00717b5051ce4e5c9c4d38de91b44de9/roles/74e9aa4660764bb0b022e7bcbc90ef86 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:54.465079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:54.509564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:54.591337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487ce65-052e-44db-9c38-9b7ee57e0b97 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:54.629153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:54.629795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.630069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.630336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.645678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:54.650871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:54.651743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:54.652529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/289] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:54.659832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:54.709569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:54.770131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dd378-5ddc-45f9-8155-fcc4976c9956 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:54.808208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:54.809242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:54.809722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:54.810172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:54.811071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:54.811550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:54.813199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:54.822223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/290] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:54.827945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:54.954080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:54.970329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331567c-b00f-42d5-b5ad-abe61b0190b8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:55.017928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:55.019366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.019827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.020293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] PATH_INFO: `/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.021948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] RBAC: Authorizing `identity:get_user(user_id=e91c6f0aa9a54b1987f1ac9eba4fee3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:55.023596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:55.025374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/291] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:54 2022] GET /identity/v3/users/e91c6f0aa9a54b1987f1ac9eba4fee3e => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:55.031075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:55.180822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1466b-2d21-408f-810c-b23b2670dcfd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:55.182294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:55.228358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:55.228975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.229249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.229517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.323704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:55.325357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7f29b-ff11-4164-8d85-ff2cc451703c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:55.452838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/292] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:55.459190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:55.553928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140390fe-1d9a-4435-86c9-ea56c75744dd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.554278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140390fe-1d9a-4435-86c9-ea56c75744dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.554466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140390fe-1d9a-4435-86c9-ea56c75744dd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.555820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/293] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:55.563287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df492b-6b20-46f8-b07e-7be9e96f36fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.563562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df492b-6b20-46f8-b07e-7be9e96f36fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.564091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df492b-6b20-46f8-b07e-7be9e96f36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.576725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:55.596871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43df492b-6b20-46f8-b07e-7be9e96f36fd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:55.598065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43df492b-6b20-46f8-b07e-7be9e96f36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:55.660877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb7a95-ff21-4be8-bfad-ba69fdd8c742 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:55.720166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/294] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:55.726028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d7af-8e9b-4b14-b336-187bae174825 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.726316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d7af-8e9b-4b14-b336-187bae174825 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.726586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26d7af-8e9b-4b14-b336-187bae174825 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.728185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/295] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:55.729480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:55.730116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.730386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.730648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.735245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:55.736174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.736485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.736752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.737642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:55.739909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/296] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:55 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:55.748315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:55.764664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:55.765822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:55.782376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:55.837805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/297] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:55.843868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:55.872871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af9f4e-c590-491d-a4fe-5ae7818e35eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:55.909066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:55.909714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.909989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.910256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.910995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=e91c6f0aa9a54b1987f1ac9eba4fee3e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:55.911303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:55.912827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cede4f-92ef-444c-90ab-11abb34137db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:55.920101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/298] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:55 2022] GET /identity/v3/role_assignments?user.id=e91c6f0aa9a54b1987f1ac9eba4fee3e&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:55.922290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ea71-3c52-42e6-bda7-5c65deabeb28 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:55.922775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ea71-3c52-42e6-bda7-5c65deabeb28 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:55.923592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ea71-3c52-42e6-bda7-5c65deabeb28 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:55.925982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/299] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:55.927545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ea71-3c52-42e6-bda7-5c65deabeb28 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:55.929981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:55.988346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91f5bc-4731-4f66-adea-5415f2674128 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.034640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.035432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.035803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.036165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.038556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.040146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999cdb4-a0ce-454f-972e-de30b09c4e6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.041965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/300] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:56.045624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.046070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.046456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.066860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.068345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:56.069763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:56.080013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ea71-3c52-42e6-bda7-5c65deabeb28 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.118733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.119458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.119798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.120473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.134372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:56.139296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:56.140150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:56.141051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:56.143424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/301] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:56.144326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.151080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/302] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:56.152615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.231515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.270579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252f33-858f-4245-ba8d-71d8f77daa87 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.277348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72251018-fcf0-4126-a193-d49d092c1801 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.307619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.308296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.308520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.308784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.309438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.310889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.314932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.315530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.315800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.316062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.316750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:56.317038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.318578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.327435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/303] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:56.328229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.329364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/304] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Aug 9 11:04:56 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:56.333590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.356374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.459915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece091-08fb-4136-9d94-8473621d96f5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.460509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333ba-7fe7-4d91-8f3d-b59a6ec5c637 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.487890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.496237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.496896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.497167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.497432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] PATH_INFO: `/v3/users/82fb962b7abf4f29a366ffef773db461` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.503371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.504003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.504294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.504597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.512075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] RBAC: Authorizing `identity:get_user(user_id=82fb962b7abf4f29a366ffef773db461)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.513851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.516001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/305] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/users/82fb962b7abf4f29a366ffef773db461 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:56.516255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:56.522130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:56.523032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:56.523846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:04:56.525016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.525087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/306] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:56.528934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.606196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.643451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07d801-b34b-4fb2-b6d1-a8ffab9b9ba7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.652191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1109bf4-74d3-44ba-b128-2a66845d056e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.678515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.679190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.679467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.679732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.680430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:56.680724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.682151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.691374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/307] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:56.696397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.696723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.697543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.698004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.698450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.699635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, user.id=82fb962b7abf4f29a366ffef773db461, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:56.700886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.701912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4498e1-6dda-48eb-a0c7-febd35c56de0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.707975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/308] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/role_assignments?user.id=82fb962b7abf4f29a366ffef773db461&scope.system=all&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:56.756197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:56.817927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56edad-d8ed-4d66-a0b4-f383b3e06aac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.855803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:56.856908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.857374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.857823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] PATH_INFO: `/v3/users/2b104df131844ab3a98ae75326113aa0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.859225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] RBAC: Authorizing `identity:get_user(user_id=2b104df131844ab3a98ae75326113aa0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:56.861076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:56.863183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/309] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:56 2022] GET /identity/v3/users/2b104df131844ab3a98ae75326113aa0 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:56.867464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84994-6719-4827-bec4-b72216a63fd5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.867750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84994-6719-4827-bec4-b72216a63fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.868021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84994-6719-4827-bec4-b72216a63fd5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.868860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:56.869570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/310] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:56.874879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:56.875158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:56.875435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:56.897434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:56.898596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:56.985811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ceab7e-8cf9-4321-bf4d-d53526bf3ddb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:56.988302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/311] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:04:57.008911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:57.011617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:57.041215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.042305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.042826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.043393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] PATH_INFO: `/v3/system/users/2b104df131844ab3a98ae75326113aa0/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.045113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b104df131844ab3a98ae75326113aa0, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:57.047006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:57.059082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/312] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Aug 9 11:04:56 2022] PUT /identity/v3/system/users/2b104df131844ab3a98ae75326113aa0/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:04:57.097123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:57.135894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e17fb6-1825-43d9-a397-7858e482c426 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:57.174450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.175661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.176139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.176717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.188827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:57.193744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:57.194657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:57.195423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/313] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:04:57.201490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:57.308297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25dbc8-5a02-4a49-ac25-25b0a7bafbe5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:57.342622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.343231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.343506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.343775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.344466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:57.344767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:57.346425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:57.355227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/314] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:57.360678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:57.464547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a784e6-1568-44eb-80a6-22c962c758fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:57.488453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:57.500551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.501844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.502115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.502377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] PATH_INFO: `/v3/users/00717b5051ce4e5c9c4d38de91b44de9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.503527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] RBAC: Authorizing `identity:get_user(user_id=00717b5051ce4e5c9c4d38de91b44de9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:57.504980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:57.506712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/315] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/users/00717b5051ce4e5c9c4d38de91b44de9 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:57.512183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:57.559738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:57.623825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d8447-044a-4140-bdb7-72d9acb63f40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:57.657297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.658196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.658635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.659065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.725072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:57.726802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:57.763802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:57.827978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/316] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:04:57.833931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:57.940473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dfacd-99c7-438d-be31-f419155823fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:57.975032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:57.975964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:57.976461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:57.976891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:57.978167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:57.979762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d9f0b-efdd-4511-8bc8-9235644e554d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:57.982889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/317] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:57.988746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:58.091313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa9790-a254-4aeb-b909-50efa36589bd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:58.125520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:58.126118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.126567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.126843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.127571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, scope.project.id=76ba1265e9274d27b43fb117daf73de6, user.id=00717b5051ce4e5c9c4d38de91b44de9) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:58.127876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:58.129356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b7667-b9d4-4904-a6f4-fc45e8ab6028 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:58.135531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/318] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:04:57 2022] GET /identity/v3/role_assignments?user.id=00717b5051ce4e5c9c4d38de91b44de9&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:58.455043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9875-f6b7-4af6-af8e-d007ea2f0135 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.455600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9875-f6b7-4af6-af8e-d007ea2f0135 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.455600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9875-f6b7-4af6-af8e-d007ea2f0135 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.456924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/319] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:58.462229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.462510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.462779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.469157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:58.482655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:58.483835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:58.560368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/320] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:58.575572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9875-f6b7-4af6-af8e-d007ea2f0135 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:58.636490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:58.680750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9875-f6b7-4af6-af8e-d007ea2f0135 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:58.714365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:58.714904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.715175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.715442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.716607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:58.718037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070fbca-65a7-445d-a425-78160da624e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:58.719750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/321] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:04:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:58.726018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:58.791660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e635-35f2-4ee6-911b-396b55e7112b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.792010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e635-35f2-4ee6-911b-396b55e7112b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.792197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907e635-35f2-4ee6-911b-396b55e7112b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.793702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/322] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:04:58.798733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.799017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.799282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.818154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:58.819344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:58.833053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0b08c-4f03-44a8-a18c-908bda4ad7c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:58.866700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:58.872438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:58.873065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:58.873331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:58.873593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:58.874222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:58.875741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:58.890712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/323] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:58.896453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/324] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Aug 9 11:04:58 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:04:58.916408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:58.980945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:59.029676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354158e-4d8d-4622-acd9-8bde837cb3ec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.068032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.068745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.069014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.069276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.079492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d904c765-2a90-420c-a47b-38af68a5cfeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:59.084209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:59.085274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:59.086251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/325] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:04:59.092471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.204214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece72a2c-567f-4526-b1d3-d72fd08bbcd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.233985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b9037-7010-4e39-b28d-22da0e273006 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.234154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b9037-7010-4e39-b28d-22da0e273006 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.235014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b9037-7010-4e39-b28d-22da0e273006 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.235964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/326] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:04:59.240938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.241131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.241370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.241558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.241651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.241834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.242104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.242773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:59.243081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:59.244479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:59.255435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/327] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:59.260761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.262282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:04:59.263829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:04:59.331130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:59.349413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/328] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:04:59.373635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.391722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad1cff-4b8d-4ab2-a182-f6d5427bc7de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.444486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.445044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:59.445153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.445396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.445671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] PATH_INFO: `/v3/users/82fb962b7abf4f29a366ffef773db461` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.447002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] RBAC: Authorizing `identity:get_user(user_id=82fb962b7abf4f29a366ffef773db461)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:59.448618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:59.455263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/329] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/users/82fb962b7abf4f29a366ffef773db461 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:04:59.456113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.504736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d58a52-2816-4341-859c-9ab19263ae8f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.539842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.540490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.540761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.541020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.552015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:04:59.557219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:04:59.558440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:04:59.559206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/330] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:04:59.564566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.565043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174876ea-a4c3-48d5-a0de-a5abff5673eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.589256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:59.602072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.602721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.602995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.603268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] PATH_INFO: `/v3/system/users/82fb962b7abf4f29a366ffef773db461/roles/74e9aa4660764bb0b022e7bcbc90ef86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.604725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=82fb962b7abf4f29a366ffef773db461, role_id=74e9aa4660764bb0b022e7bcbc90ef86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:59.606119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:59.617020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/331] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Aug 9 11:04:59 2022] PUT /identity/v3/system/users/82fb962b7abf4f29a366ffef773db461/roles/74e9aa4660764bb0b022e7bcbc90ef86 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:04:59.687511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311f276-d096-465c-9362-ca479b5e0e16 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.696984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:04:59.728084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.729029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.729304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.729592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.730314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:04:59.730913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:59.732350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:59.742747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/332] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:59.748607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:04:59.865848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1d0f7-d0de-4855-a2be-a9b86a2dce8a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:04:59.901722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:04:59.902392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:04:59.902630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:04:59.902900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] PATH_INFO: `/v3/users/2b104df131844ab3a98ae75326113aa0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:04:59.904027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2b104df131844ab3a98ae75326113aa0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:04:59.905583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef73c7-e4bc-4ff8-a58b-69d7c10ab1c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:04:59.907435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/333] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/users/2b104df131844ab3a98ae75326113aa0 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:04:59.914077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:00.050896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292eae3-4c69-473e-9eaf-b765cadf45f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:00.092238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:00.096930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:00.097200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:00.097476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:00.098289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] RBAC: Adding query filter params (role.id=3a91cfd137cf4404b2ed6d52380e69d5, user.id=2b104df131844ab3a98ae75326113aa0, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:00.098555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:00.100040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53263-c33e-48dd-9f6a-6f80b008ebde None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:00.112619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/334] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 9 11:04:59 2022] GET /identity/v3/role_assignments?user.id=2b104df131844ab3a98ae75326113aa0&scope.system=all&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:01.159723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e32e3a-4875-4665-9313-f03365de5169 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:01.160752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e32e3a-4875-4665-9313-f03365de5169 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:01.161226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e32e3a-4875-4665-9313-f03365de5169 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:01.162766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:01.168668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:01.169178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:01.169178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:01.191088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:01.192363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:01.220502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:01.274385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:01.304644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e32e3a-4875-4665-9313-f03365de5169 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:01.367185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:01.425274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e32e3a-4875-4665-9313-f03365de5169 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:01.466681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:01.467308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:01.467576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:01.467839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:01.485173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:01.490696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:01.491588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:01.492396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:01.493175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/337] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:01.498143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:01.664426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e0c59-1c89-4c4a-9666-51e59a7bdf18 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:01.710517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:01.711038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:01.711308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:01.711573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:01.712229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:01.712551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:01.713922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470a7c6-aaef-4ec5-8bca-0174c70e17ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:01.728408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/338] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:01.734051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:01.846687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b878c-5c06-4c76-a1fa-b0cb872f6dbb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:01.865129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:01.886243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:01.886849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:01.887122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:01.887388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:01.980083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:01.981516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:02.057228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb59ae-86c8-4b10-9c1d-2fad80072886 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.057888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb59ae-86c8-4b10-9c1d-2fad80072886 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.058349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb59ae-86c8-4b10-9c1d-2fad80072886 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.062040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/339] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:02.066644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.067209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.067670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.113418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:02.115011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:02.118713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/340] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:02.133661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.137511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:02.170348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:02.269090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/341] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:02.316489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce67cb-897f-47c3-9ecf-f34bb8d48b7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:02.359964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:02.360206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.360911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.361190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.361460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.362603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:02.364055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:02.365716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/342] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:02.371162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.455805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:02.477793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d8206-ace0-4f00-8728-0b316e72b3b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:02.545490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:02.546040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.546312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.546578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.562176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:02.563820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:02.565562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/343] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:02.574253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.601338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296fc5-2aea-457f-b436-b0a067d94849 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:02.679577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:02.680205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.680495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.680755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.701334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-23c72c31-5361-487c-a047-b93cd6650ffb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:02.706519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:02.707371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:02.708139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/344] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:02.714990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f83a3-adc0-4895-bc63-7749e171f73a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.715262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f83a3-adc0-4895-bc63-7749e171f73a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.715526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f83a3-adc0-4895-bc63-7749e171f73a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.716872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/345] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:02.717589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f83a3-adc0-4895-bc63-7749e171f73a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.735876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:02.763196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763417a-08c4-4dbe-9e11-5470ec5bee39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:02.802168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:02.802696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.802962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.803224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.803939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=13abcee6071741f590ffaf32659ac1c6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:02.804242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:02.805915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4f64a-942e-4401-ad0d-92ff849aca21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:02.812023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/346] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:02.813458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff59e-cae2-4894-8e8e-50620cdb7a48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.813958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff59e-cae2-4894-8e8e-50620cdb7a48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.814229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff59e-cae2-4894-8e8e-50620cdb7a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.833996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19dff59e-cae2-4894-8e8e-50620cdb7a48 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:02.835226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19dff59e-cae2-4894-8e8e-50620cdb7a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:02.838586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:02.842140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f83a3-adc0-4895-bc63-7749e171f73a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:02.879170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:02.880030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.880503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.880910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.881719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:02.882158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:02.883729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301e83-cf3a-4aff-9bbe-aa0428e90a49 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:02.894270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/347] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:02.895840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122703f-2da1-4dd9-ac26-20b1aa860e4f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.896639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122703f-2da1-4dd9-ac26-20b1aa860e4f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.897047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122703f-2da1-4dd9-ac26-20b1aa860e4f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.898867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/348] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:02.899824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122703f-2da1-4dd9-ac26-20b1aa860e4f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:02.913302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/349] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:02.914057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:02.914333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:02.914591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:02.948085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:02.949378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:02.993539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.029858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/350] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:03.030712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.036242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122703f-2da1-4dd9-ac26-20b1aa860e4f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.091831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.092751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.093171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.093583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.119230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.148320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b4271-32dc-4d00-b07e-29dfa2df5f19 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.180641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.181370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.181697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.182025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.194747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:03.199466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:03.200597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:03.201589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:03.202783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/351] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:03.203823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.210306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.211982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.291828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.323370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4140b5-7966-40dc-bb6c-e4234a9a0933 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.330053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/352] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:03.331272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.376824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.386031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.386768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.387044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.387310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.401759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:03.406642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:03.407610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 11:05:03.408437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:03.413877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/353] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:03.415028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.446756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee932-169a-4bc7-bda7-943c7685b3b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.483991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.484685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.484961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.485226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.486578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:03.486880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.488257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.497214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/354] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:03.498060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.532212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.567848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62a357-bf5d-48df-99af-3d7f7527c8ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.607321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fbd7-a5e5-4a7b-a5b2-e46d663b3748 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.608888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.609788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.610274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.610770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.612291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.614014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c361489-5254-4f98-afa6-923880eda405 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.618858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/355] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:03.619174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3188-263a-468d-a90f-266197a1fdc5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.620126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3188-263a-468d-a90f-266197a1fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.620126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b3188-263a-468d-a90f-266197a1fdc5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.621373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/356] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:03.621966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3188-263a-468d-a90f-266197a1fdc5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.652114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.652683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.652954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.653214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.653883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:03.654194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.655577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.664955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/357] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:03.665849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.717578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.771455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b3188-263a-468d-a90f-266197a1fdc5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.775147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:03.784211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5e986-5cf2-4fdb-a3e0-e53878b46522 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:03.815836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.816540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.816976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.817300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.830165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:03.831175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.831644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.832102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.833489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.835117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8265f-0ffe-4004-970c-dab50f650e8b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.840391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/358] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:03.841812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:03.842285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:03.842743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:03.864450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:03.866011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:03.906058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:03.909158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:03.943724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/359] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:03.945090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:03.994773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.056470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/360] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:04.057679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.064207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.074224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23bfa8-9cdd-45d2-bec1-c678128de238 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.110138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.110851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.111385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.111461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] PATH_INFO: `/v3/users/82fb962b7abf4f29a366ffef773db461` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.113642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] RBAC: Authorizing `identity:get_user(user_id=82fb962b7abf4f29a366ffef773db461)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.115073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.117032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/361] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:05:03 2022] GET /identity/v3/users/82fb962b7abf4f29a366ffef773db461 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.117822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.186433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f6c5-9ca3-4c66-9b2b-fbe972237277 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.229346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.229949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.230219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.230481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.232075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=13abcee6071741f590ffaf32659ac1c6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:04.232075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.233544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.239830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/362] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.240667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.265685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f939fd-3e88-4199-8d0f-4adc27dc9849 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.272688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.323245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.323930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.324152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.324449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.339388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:04.346840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:04.347772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:04.348593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/363] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:04.349820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.376090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfcc32-b09e-42cf-bdd8-ce54372a4961 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.382261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.422566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.423360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.423647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.423914 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.425107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.427382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2ef4f-0259-4b79-9ddc-d251f1a88316 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.429079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/364] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.430273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e35d7-02f1-46d8-816e-473821326a7e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.430537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e35d7-02f1-46d8-816e-473821326a7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.430811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e35d7-02f1-46d8-816e-473821326a7e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.432162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/365] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:04.433830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e35d7-02f1-46d8-816e-473821326a7e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.520416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e01e-b082-42c9-8c9f-836864bbda31 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.554936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.566647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.566875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.566972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.567227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.567984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] RBAC: Adding query filter params (role.id=74e9aa4660764bb0b022e7bcbc90ef86, user.id=82fb962b7abf4f29a366ffef773db461, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:04.568309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.568873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e35d7-02f1-46d8-816e-473821326a7e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.569767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.585261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/366] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/role_assignments?user.id=82fb962b7abf4f29a366ffef773db461&scope.system=all&role.id=74e9aa4660764bb0b022e7bcbc90ef86 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.586358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.609492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.610546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.611020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.611482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.612982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:04.613502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.615081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533366ff-6003-4261-92e7-2fec29c333c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.627059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/367] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.628685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.629170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.629658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.649629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:04.651236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:04.687070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.717377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1ebc-02ce-42d4-a8bd-62cc607203c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.744373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/368] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:04.745474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.758420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.760028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.760538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.761026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.762344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.764043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.765989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/369] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:04.773657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.852744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:04.873274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06dde-a765-450a-93bd-767a610596d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.888958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7715a73-3318-4b06-8d72-135182ded2da None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:04.915645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.916538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.917109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.917569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.944680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:04.945285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:04.945554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:04.945830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:04.946547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=76ba1265e9274d27b43fb117daf73de6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:04.946848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:04.948289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:04.959017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/370] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:04.960153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:04.982135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.028331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.029835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.122280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.124451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51791b-1582-4760-8bb6-82040e89de1e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.144867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/371] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:05.151120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.193902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.194809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.195273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.195670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.211300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:05.216848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:05.218010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:05.218882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:05.221390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/372] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:05.242545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.271025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412d4a-e0d5-4ade-83bc-d2f26999e98f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.310901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.311457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.311721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.311980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.313189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.314716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.318689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/373] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:05.323765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.325279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.357440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4f47d-e776-4aa8-9289-908cb8a4c4b9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.386677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.394939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.395440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.395805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.396520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.397261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:05.397653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.399306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.409693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/374] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:05.415422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.443541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6dd06-d506-42cf-b54b-7986409ba7bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.480711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.481621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.482052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.482450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.483691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.485525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.487661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/375] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:05.493691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.522059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f15b7c-da3e-49e7-be35-98c69caceab1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.556667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.560328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.561268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.561268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.561470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.602606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc75f-c3d5-440d-a274-dd9da99b749d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.620857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.636888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.638443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.640464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:05.641185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.642424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.642791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.643614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=76ba1265e9274d27b43fb117daf73de6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:05.644011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:05.645573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a64d-1ce1-42b5-bbd4-8a74e2715d56 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:05.652006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/376] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:05.664857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919093a6-3f1a-4171-9ad8-5fb5b9860b1c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.665196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919093a6-3f1a-4171-9ad8-5fb5b9860b1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.665413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919093a6-3f1a-4171-9ad8-5fb5b9860b1c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.667157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/377] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:05 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:05.672938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:05.673235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:05.673472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:05.701100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:05.702301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:05.774612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/378] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:05:05.780230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.817550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/379] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:05.841854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:05.853557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:05.983294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466caa8e-9a51-4f17-b81e-567009049c08 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:05.994078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117c1cd-f11d-41a7-b819-4f753cdb5d32 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.009376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:06.042800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.043455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.043724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.044001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.059259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.059893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.060169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.061543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] PATH_INFO: `/v3/projects/69344eba9b82497c957a2d6fd891ec79` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.063415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] RBAC: Authorizing `identity:get_project(project_id=69344eba9b82497c957a2d6fd891ec79)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.065342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.066988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/380] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/projects/69344eba9b82497c957a2d6fd891ec79 => generated 344 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:06.071884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:06.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:06.077440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:06.078206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:06.078794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.079903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/381] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:06.098440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.219658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71fe8-2af1-49c6-a8a1-ec580665540f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.228772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:06.257053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.257682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.257956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.258233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] PATH_INFO: `/v3/groups/b4f107eddbea44658e36b32f61e21bce` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.264134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] RBAC: Authorizing `identity:get_group(group_id=b4f107eddbea44658e36b32f61e21bce)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.265605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.268020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/382] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:06 2022] GET /identity/v3/groups/b4f107eddbea44658e36b32f61e21bce => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:06.274184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.345376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd7170-89c6-47ce-8cef-edff342a38cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.397364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.398470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.398764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.399034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.399991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:06.400323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.401940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.411607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/383] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:06.417250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.463756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:06.493021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fba2-1a9a-4ef5-a94d-1400d51acc39 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.540308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ec67d-e01c-48ee-acdc-d5c092b0ac58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.569030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.569650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.569924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.570190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.570921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] RBAC: Adding query filter params (group.id=b4f107eddbea44658e36b32f61e21bce, role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=69344eba9b82497c957a2d6fd891ec79) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:06.571221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.572674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.577039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.577674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.578013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.578284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.579409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/384] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:06 2022] GET /identity/v3/role_assignments?group.id=b4f107eddbea44658e36b32f61e21bce&scope.project.id=69344eba9b82497c957a2d6fd891ec79&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:06.579626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.581117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.583039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/385] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:06 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:06.588745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.629277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:06.714075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bf2fa-5c34-44d9-a2dd-ba7e290de885 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.752657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.753337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.753610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.753884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.755039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.756525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acea10c-34bc-4dcd-afdf-9a6b84675be3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.758169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/386] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:06 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:06.764518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:06.880718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27e54-a316-482f-85f9-7be5f4ba38bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:06.882190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:06.917919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:06.918663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:06.919013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:06.919354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/groups/195ca369a915462c9281800771633759/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:06.921429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, group_id=195ca369a915462c9281800771633759, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:06.923111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80ba3f-74b3-4b2d-9774-2a745df4c14d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:06.938704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/387] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 9 11:05:06 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/groups/195ca369a915462c9281800771633759/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:07.336155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9bc0-e2b2-4d91-817c-01d3bd7baa56 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.336479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9bc0-e2b2-4d91-817c-01d3bd7baa56 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.336710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab9bc0-e2b2-4d91-817c-01d3bd7baa56 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.338349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/388] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:07.343285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a285ba4-b25b-468d-8305-59c1fd41d1bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.343813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a285ba4-b25b-468d-8305-59c1fd41d1bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.344294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a285ba4-b25b-468d-8305-59c1fd41d1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.368126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a285ba4-b25b-468d-8305-59c1fd41d1bc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:07.371292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a285ba4-b25b-468d-8305-59c1fd41d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:07.479449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/389] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:07.514774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9bc0-e2b2-4d91-817c-01d3bd7baa56 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:07.609085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2fe-e21e-49b2-ae5d-405d126f20fa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.609263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2fe-e21e-49b2-ae5d-405d126f20fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.609506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2fe-e21e-49b2-ae5d-405d126f20fa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.611196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/390] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:07.617842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.618122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.618393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.639190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:07.643032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:07.645534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:07.649859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab9bc0-e2b2-4d91-817c-01d3bd7baa56 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:07.728805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/391] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:07.738343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:07.738999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.739266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.739523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.752101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:07.765254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:07.770151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:07.771722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:07.772992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/392] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:07.778439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:07.793496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:07.884669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eb855-6276-4832-8093-4c10dc398bc6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:07.952789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:07.956289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:07.956289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:07.956289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:07.976071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:07.986802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:07.987716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:07.988711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/393] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:07.997649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.016091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0f0c-2cbc-4e11-af3b-3148f2ed431f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.067697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.068696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.069200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.069694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.070565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:08.071068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.072692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.081808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/394] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:08.088063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.124192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.166848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.192682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7e4e-3c3a-4137-a97d-e96bdad7dfd9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.237222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.237868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.238144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.238412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.239477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:08.239779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.242728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.260309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/395] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:08.260309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.277585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943298c-31f5-450c-9ddf-ad30d481ed10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.333900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.334648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.334995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.335336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.339021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.339021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.340756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/396] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:08 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:08.346281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.424490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f5993-00bb-4cdf-a727-207b4c54ac89 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.467466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.468106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.468407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.468674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.469852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.471657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ec5df-564b-465d-a4ab-157487c67775 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.473644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/397] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:08 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:08.474792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51842-b793-464a-b79e-b958b068b16c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.475065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51842-b793-464a-b79e-b958b068b16c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.475325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51842-b793-464a-b79e-b958b068b16c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.476702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/398] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:08.480370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.480996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51842-b793-464a-b79e-b958b068b16c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.500898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c499a-70d4-4a98-bf40-bddbea06aa5a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.503558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.542565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.543974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.544501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.544980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.546342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.548842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb6997-eec3-47db-a994-8e82ab560026 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.551071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/399] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:08 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:08.552458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.552940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.553425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.572623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:08.574149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:08.627027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51842-b793-464a-b79e-b958b068b16c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.647103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/400] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:08.648419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.663081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.663709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.663978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.664237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.665459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.667038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.669250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/401] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:08 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:08.681338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.757970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.777215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff609-6d83-41ee-8308-92f99bea7bf6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.798192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:08.822328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.822557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dcf4d-791a-4c71-9d90-da88b4b2695b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:08.823488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.824203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.825609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6/groups/195ca369a915462c9281800771633759/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.827997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] RBAC: Authorizing `identity:create_grant(project_id=13abcee6071741f590ffaf32659ac1c6, group_id=195ca369a915462c9281800771633759, role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.829840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.848679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/402] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 9 11:05:08 2022] PUT /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6/groups/195ca369a915462c9281800771633759/roles/7f08073ba42c48ff8060c007983602f6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:08.850153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:08.867947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:08.868744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:08.869134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:08.869783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/groups/195ca369a915462c9281800771633759/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:08.873837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, group_id=195ca369a915462c9281800771633759, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:08.876835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:08.902824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/403] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 9 11:05:08 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/groups/195ca369a915462c9281800771633759/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:08.995384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102949c-08f9-46a0-96b1-4afa6253d687 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.035524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:09.036234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.036542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.036810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.049577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:09.054549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:09.055492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:09.056393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/404] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:09.063077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:09.182818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231d3ad-2288-417e-87d9-93bd7f070257 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.220228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:09.220978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.221252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.221556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.222225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:09.222517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:09.223909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:09.233279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/405] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:09.238785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:09.243272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:09.351142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e426f-35eb-4de8-aa15-22cd2050f51e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.386163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:09.386838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.387108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.387415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.388578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:09.390003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:09.391908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/406] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:09 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:09.397511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:09.404356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:09.528112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc2527-6e18-4265-a684-81e5bfd2e01d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.571231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:09.572244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.572741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.573218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.574542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:09.576260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8570f-59ac-4846-b101-f0ad27828229 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:09.582934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/407] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:09 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:09.584520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:09.642189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516756-8d0c-4822-8dda-e9d6dbb8d05f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.642842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516756-8d0c-4822-8dda-e9d6dbb8d05f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.643489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516756-8d0c-4822-8dda-e9d6dbb8d05f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.646840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/408] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:09.651420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.651932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.652450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.677819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:09.682570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:09.684027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:09.705227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2031aa9-a47f-4f75-b1ae-109267ab9eb2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.741743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:09.742402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.742674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.742934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6/groups/195ca369a915462c9281800771633759/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.746953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] RBAC: Authorizing `identity:create_grant(project_id=76ba1265e9274d27b43fb117daf73de6, group_id=195ca369a915462c9281800771633759, role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:09.749064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10214-be40-49fc-bdc2-08745759592d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:09.767547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/409] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 9 11:05:09 2022] PUT /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6/groups/195ca369a915462c9281800771633759/roles/7f08073ba42c48ff8060c007983602f6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:09.774478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7018c-1816-4aa0-a84a-bbc0c2685be8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.774478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7018c-1816-4aa0-a84a-bbc0c2685be8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.774478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7018c-1816-4aa0-a84a-bbc0c2685be8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.778429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/410] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:09.792049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:09.792399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:09.792609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:09.818162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:09.824583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:09.825298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/411] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:09.825948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:09.861922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:09.991559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67098219-b281-49f7-8cf1-cfbbeefaced8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:09.999076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/412] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:10.027031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.035541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.036209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.036504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.036772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.050456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:10.055158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:10.056042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:10.056870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/413] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:10.063952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.109103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.132137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.159590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f3220-b317-4d4f-ba32-e46d5862b270 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.197198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.197746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.198016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.198280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.206824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ff70-647d-4bbc-9b1e-1a6ab96d7c9e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.210208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:10.216037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:10.216939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:10.217711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/414] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:10.224791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.275289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.275964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.276240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.276524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.277898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:10.278399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.279810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:10.300060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/415] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:10.306301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.431997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a5926-de2f-497a-9e77-c6288f7418c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.448445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb84e0-beaa-43ea-a4ad-5c68ab0ced33 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.471860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.475196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.489380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.490040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.490313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.490577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] PATH_INFO: `/v3/groups/b4f107eddbea44658e36b32f61e21bce` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.492904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] RBAC: Authorizing `identity:get_group(group_id=b4f107eddbea44658e36b32f61e21bce)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.494367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:10.496286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/416] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/groups/b4f107eddbea44658e36b32f61e21bce => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:10.497616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.498222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.498501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.498771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.499439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:10.499739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.502532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:10.503548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.518672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/417] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:10.524540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.670583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa3e57-3e27-41b4-a477-115f7c20ce99 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.709237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.710684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.711120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.711531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] PATH_INFO: `/v3/projects/69344eba9b82497c957a2d6fd891ec79` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.712979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] RBAC: Authorizing `identity:get_project(project_id=69344eba9b82497c957a2d6fd891ec79)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.714830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:10.717526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/418] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/projects/69344eba9b82497c957a2d6fd891ec79 => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:10.722891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.725928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc093f7-c708-4331-a6ee-3d71cd3a5a5a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.784132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.785079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.785571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.786028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.840355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.843619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:10.861574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.863239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:10.909166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191e724-e015-4299-92d4-143a28fc3157 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:10.966110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/419] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:05:10.971682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:10.983027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:10.983978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:10.984508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:10.984985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] PATH_INFO: `/v3/projects/69344eba9b82497c957a2d6fd891ec79/groups/b4f107eddbea44658e36b32f61e21bce/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:10.989264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] RBAC: Authorizing `identity:create_grant(project_id=69344eba9b82497c957a2d6fd891ec79, group_id=b4f107eddbea44658e36b32f61e21bce, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:10.990998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:11.013450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/420] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 9 11:05:10 2022] PUT /identity/v3/projects/69344eba9b82497c957a2d6fd891ec79/groups/b4f107eddbea44658e36b32f61e21bce/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:11.082397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:11.098475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b82c94-03c7-4ba1-95bd-f322b763e3ae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.138432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:11.139062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.139336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.139607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.140755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:11.142258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c5eff-a749-43fb-b7d3-31515e761072 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:11.144301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/421] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:10 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:11.153513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.297434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba471b18-baac-43ce-9f0a-06087d47e479 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.312065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4052e-245f-4314-bbc7-d88d04c66200 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.312065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4052e-245f-4314-bbc7-d88d04c66200 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.312328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4052e-245f-4314-bbc7-d88d04c66200 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.314297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/422] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:11.319285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea919-b18e-42d8-a365-fde9e2ad201b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.319565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea919-b18e-42d8-a365-fde9e2ad201b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.319840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaea919-b18e-42d8-a365-fde9e2ad201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.325061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:11.347362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3eaea919-b18e-42d8-a365-fde9e2ad201b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:11.348659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:11.349171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3eaea919-b18e-42d8-a365-fde9e2ad201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:11.349275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.349551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.349812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.350984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:11.352489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:11.354258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/423] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:11.359880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.476654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/424] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:11.512182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be1f8d-761c-4cfc-b4f8-637244b5d33d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.512731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be1f8d-761c-4cfc-b4f8-637244b5d33d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.513141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be1f8d-761c-4cfc-b4f8-637244b5d33d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.514594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/425] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:11.519646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.520078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.520525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.541423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:11.542764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:11.543349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dc4db-0a76-46b3-bb54-60d879745550 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.579393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:11.580240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.580691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.581104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.582067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=13abcee6071741f590ffaf32659ac1c6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:11.582627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:11.584076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:11.590497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/426] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:11.591297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.632298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/427] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:11.653094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.672327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:11.682850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:11.786304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fa430-3d99-4a38-ac9d-a6df49a67e32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.808170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16262f-fae7-46c0-882e-4083c5d04884 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.824828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:11.825459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.825730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.825995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.839375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:11.846597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:11.847481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:11.848285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:11.849240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/428] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:11.855530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.859685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:11.860443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:11.860718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:11.860985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:11.872859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:11.877778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:11.879095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:11.885120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/429] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:11.890569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:11.973883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d537-84c2-4ee8-85d8-a28e35a4bbf8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:11.987693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:11.995012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.015001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.015771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.016141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.016675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.017447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:12.017845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.019383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.025498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a565d5-02ca-439c-9c62-ecb813374d26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.032681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/430] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:12.035633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.064786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.065759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.066168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.066666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.067936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:12.068406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.070126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.080965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/431] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:12.089105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.162492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056a0f5-b40e-41b2-8aa8-bfcddda5af79 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.200761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.201540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.201912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.202289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.212968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39a27e-cc7f-4228-bd27-0b8305e655e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.248603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.249328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.250539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.251460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.252281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.257288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.283182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.285097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7510d-3d4d-4ab4-bc96-9a43c06a0961 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.328595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.330381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579ef7d-2baa-454a-9e6a-0df8042f3daf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.409096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/432] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:12.410642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09339d47-bc40-481d-86e5-850fb99ea829 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.410922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09339d47-bc40-481d-86e5-850fb99ea829 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.411196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09339d47-bc40-481d-86e5-850fb99ea829 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.413208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/433] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:12.417123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09339d47-bc40-481d-86e5-850fb99ea829 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.447660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/434] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:12.448940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.449213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.449476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.467660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:12.468940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:12.495890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.500480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.533647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09339d47-bc40-481d-86e5-850fb99ea829 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.537187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/435] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:12.538069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.577177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.577810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.578088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.578355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] PATH_INFO: `/v3/projects/13abcee6071741f590ffaf32659ac1c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.579507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] RBAC: Authorizing `identity:get_project(project_id=13abcee6071741f590ffaf32659ac1c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.580972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.582731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/436] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/projects/13abcee6071741f590ffaf32659ac1c6 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:12.585081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.651663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12afb66-d16f-47a2-978d-ede54b2cb6f5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.688154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.688940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.689305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.689574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.690420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26030a-c792-4cd7-908d-30353f563b95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.690874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.692096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.693885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/437] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:12.694448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.725646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.732046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.732712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.732985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.733249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.742152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.745236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:12.750323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:12.751258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:12.752163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/438] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:12.752842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.809026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6625b1f7-7e6b-4888-86f8-8521b7bd08a2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.849081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.849649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.849902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.850176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.851529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.852792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.859112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/439] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:12.859112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.862973 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8fa4d-236a-4989-ba73-6e68971e862e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.897564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:12.898272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:12.898545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:12.898808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:12.899895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:12.901367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:12.902904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/440] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:12.903647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:12.975525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:12.979506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695098a-0771-484d-a240-47b48ee02447 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:12.985921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.018272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.018871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.019138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.019402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.020050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:13.020393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.020633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dd9-1165-4d61-bcc1-c979df27defe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.022145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.031005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/441] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.031621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.060932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.061587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.061857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.062358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.062855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=13abcee6071741f590ffaf32659ac1c6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:13.063153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.064794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.070640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/442] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:12 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=13abcee6071741f590ffaf32659ac1c6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.071405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.152612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2c87-81d0-4776-91c0-1fcece534fbd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.191326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.191935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.192240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.192516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.193264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=3a91cfd137cf4404b2ed6d52380e69d5, scope.project.id=76ba1265e9274d27b43fb117daf73de6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:13.193565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.195074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184c55-67e6-46de-8fc6-4ca0f5507d87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.201887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/443] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=3a91cfd137cf4404b2ed6d52380e69d5 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.217924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbafee2-c8f9-4f7c-951b-1440acae5884 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.258111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.266071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.266783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.266783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.267437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.351676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.353273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.363352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7777d-45e1-46c6-b09f-bcfe4b57b365 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.363632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7777d-45e1-46c6-b09f-bcfe4b57b365 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.363901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7777d-45e1-46c6-b09f-bcfe4b57b365 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.365885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/444] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:13.370816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.371131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.371354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.396807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:13.398026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:13.423364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.481041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/445] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:13.482789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/446] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:13.489289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.505134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.544875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.599934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ede2e3-4447-424c-a938-675f29d1872b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.635775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb343a-3c39-42c8-a40f-a5270ba21e7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.642607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.643229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.643511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.643770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] PATH_INFO: `/v3/projects/76ba1265e9274d27b43fb117daf73de6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.644916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] RBAC: Authorizing `identity:get_project(project_id=76ba1265e9274d27b43fb117daf73de6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.646653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.648584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/447] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/projects/76ba1265e9274d27b43fb117daf73de6 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.654377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.678878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.679789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.680060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.680488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.682729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.692368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:13.698542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:13.699477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:13.700259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/448] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:13.707963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.759285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26673683-897e-4a7d-b1c9-ed36199bd111 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.792712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.793324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.793591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.793850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] PATH_INFO: `/v3/groups/195ca369a915462c9281800771633759` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.795021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] RBAC: Authorizing `identity:get_group(group_id=195ca369a915462c9281800771633759)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.796437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.798060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.803145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/449] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/groups/195ca369a915462c9281800771633759 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.804023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.807585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968466-d3e8-49a6-a387-941b829ab02a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.840549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.841149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.841417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.841680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.842336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:13.842628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.844002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.852156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/450] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:13.857683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:13.893055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:13.902083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095f27-0898-44b9-8d61-9931c0056643 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.936352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.936944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.937213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.937478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:13.938220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] RBAC: Adding query filter params (group.id=195ca369a915462c9281800771633759, role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=76ba1265e9274d27b43fb117daf73de6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:13.938553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:13.940155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:13.945843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/451] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/role_assignments?group.id=195ca369a915462c9281800771633759&scope.project.id=76ba1265e9274d27b43fb117daf73de6&role.id=7f08073ba42c48ff8060c007983602f6 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:13.960241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277d274-c2e5-429c-a7da-42a72342b404 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:13.998418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:13.999043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:13.999311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:13.999570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:14.027247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:14.073885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:14.075260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:14.178483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/452] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:05:14.184602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:14.284075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2e8a0-1079-4297-921f-d7736044d57f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:14.315879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:14.316516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:14.316789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:14.317055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] PATH_INFO: `/v3/projects/69344eba9b82497c957a2d6fd891ec79` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:14.318146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] RBAC: Authorizing `identity:get_project(project_id=69344eba9b82497c957a2d6fd891ec79)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:14.319559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:14.321179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/453] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:05:14 2022] GET /identity/v3/projects/69344eba9b82497c957a2d6fd891ec79 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:14.326946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:14.382573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:14.428627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4f229-4151-4568-8bae-477a717f3e90 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:14.463855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:14.464483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:14.464751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:14.465012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] PATH_INFO: `/v3/groups/b4f107eddbea44658e36b32f61e21bce` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:14.466231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] RBAC: Authorizing `identity:get_group(group_id=b4f107eddbea44658e36b32f61e21bce)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:14.467664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d727794-bc49-4bae-8d9a-1453c344bde8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:14.469208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/454] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 9 11:05:14 2022] GET /identity/v3/groups/b4f107eddbea44658e36b32f61e21bce => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:14.476179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:14.506794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:14.581724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffc756-2ce9-4f08-b6e9-e9f4337f6be9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:14.616740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:14.617360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:14.617615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:14.617953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:14.618712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] RBAC: Adding query filter params (group.id=b4f107eddbea44658e36b32f61e21bce, role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=69344eba9b82497c957a2d6fd891ec79) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:14.619098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:14.620529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdcebf-b6ca-4922-a702-7de6ed89a403 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:14.626375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/455] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 9 11:05:14 2022] GET /identity/v3/role_assignments?group.id=b4f107eddbea44658e36b32f61e21bce&scope.project.id=69344eba9b82497c957a2d6fd891ec79&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:17.022131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746596-1cd6-49d7-818c-fa82e993e513 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.022694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746596-1cd6-49d7-818c-fa82e993e513 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.022694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746596-1cd6-49d7-818c-fa82e993e513 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.024260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/456] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:17.029369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bbd9-0679-4961-b07f-5c7860a39771 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.029804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bbd9-0679-4961-b07f-5c7860a39771 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.029898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4bbd9-0679-4961-b07f-5c7860a39771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.052207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10b4bbd9-0679-4961-b07f-5c7860a39771 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:17.053430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10b4bbd9-0679-4961-b07f-5c7860a39771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:17.142537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/457] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:17.202593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746596-1cd6-49d7-818c-fa82e993e513 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.259126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb2a9-339b-4d45-8fd1-1d5826ff54c1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.259446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb2a9-339b-4d45-8fd1-1d5826ff54c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.259667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bb2a9-339b-4d45-8fd1-1d5826ff54c1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.261331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/458] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:17.265763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cd8a-d26a-4187-9220-879eef4726b7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.266036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cd8a-d26a-4187-9220-879eef4726b7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.266431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cd8a-d26a-4187-9220-879eef4726b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.290451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a01cd8a-d26a-4187-9220-879eef4726b7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:17.291620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a01cd8a-d26a-4187-9220-879eef4726b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:17.325237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.372804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746596-1cd6-49d7-818c-fa82e993e513 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:17.399960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.403394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/459] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 11:05:17.404834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d645-7a83-4200-8c68-907f65196429 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.405125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d645-7a83-4200-8c68-907f65196429 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.405398 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4d645-7a83-4200-8c68-907f65196429 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.410900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/460] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:17.412369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.412663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.413037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.417290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:17.418162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.418649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.419116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.430663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:17.432246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:17.432835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:17.434405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:17.447765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:17.448916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:17.449958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:17.449958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.449958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:17.451238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/461] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:17.452099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.527395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/462] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:17.535408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.575026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ff361-e0cf-4d36-bc98-3c8f8dea0a10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:17.621218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:17.622034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.622719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.623110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.634250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.637587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:17.639397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:17.658868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:17.659780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:17.660676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:17.660676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.660676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:17.662783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/463] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:17.663872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.665621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22d29a-43ff-46db-ae5e-43f87f240841 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:17.693888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.721135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:17.722109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.722450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.722786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.723499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:17.723860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:17.725412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:17.741811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/464] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:17.742546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.776094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02334ad-0ad8-469d-9945-bd82a1e415aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:17.817915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:17.818472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.818749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.819015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.835787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:17.837717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:17.857933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:17.858882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:17.859714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:17.859714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:17.859714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:17.862570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/465] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:17.863861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.874224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903ec1-393b-430f-811f-16ebadd14bc5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:17.925021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:17.926058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:17.926527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:17.926985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:17.927821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:17.928399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:17.932534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:17.941688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.948924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/466] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:17.950048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:17.970698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:17.994852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dc8db-bd97-4078-8826-cb50204b6f94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.031569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.032468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.032666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.032978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.033680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.035210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.059495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4aca-0251-4496-84dc-bc43576766fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.064438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/467] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:17 2022] POST /identity/v3/users => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:18.064438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.107424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.108882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.109463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.110055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.111029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:18.111777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.114518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb3b3c-9368-4427-832e-197e92d96936 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.138433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/468] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:18.138433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a47e6-b7f1-4cfa-bf73-44456c3793e2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.138433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a47e6-b7f1-4cfa-bf73-44456c3793e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.138803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a47e6-b7f1-4cfa-bf73-44456c3793e2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.141796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/469] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:18.142552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a47e6-b7f1-4cfa-bf73-44456c3793e2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.205035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f35c55-52b8-40f0-b551-97883ebfa671 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.238391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:18.246726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.247330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.247607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.247882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.248552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.249988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3310ac1-9186-4060-9951-0789ea222051 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.258305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:18.278809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/470] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:18.280066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350004f-6f87-4897-9038-da19572b9e53 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.280359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350004f-6f87-4897-9038-da19572b9e53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.280629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350004f-6f87-4897-9038-da19572b9e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.295257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a47e6-b7f1-4cfa-bf73-44456c3793e2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.299569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f350004f-6f87-4897-9038-da19572b9e53 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:18.300815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f350004f-6f87-4897-9038-da19572b9e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:18.338696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.339412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.339775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.340048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.341385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.369881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/471] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:18.377951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/472] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:18.405273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.509385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d66c8b-8458-4ca0-ba5b-cb2815f1cdec None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.509756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d66c8b-8458-4ca0-ba5b-cb2815f1cdec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.509940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d66c8b-8458-4ca0-ba5b-cb2815f1cdec None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.511602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/473] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:18.517707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.518013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.518246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.525863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7697bf-d978-404a-bee3-869b3004c827 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.538518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:18.539834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:18.544694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:18.563544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.564311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.565538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.565806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.578481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.580385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:18.597551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:18.598483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:18.599756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:18.599756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:18.599756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:18.599756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/474] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:18.605510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.631229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/475] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:18.658218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.724027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:18.736122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514d4ab-5cb1-453b-8010-4f81e7223181 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.776087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.776738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.777015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.777341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.778032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:18.778504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.779947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.792610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf808-279c-4b43-80e6-5e58f15b510a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:18.792946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/476] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:18.814811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.835030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:18.835645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:18.835923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:18.836189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:18.836921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:18.839095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:18.849120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 11:05:18.849120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 11:05:18.849120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [parameters: {'id': 'a2ffaa74d754448a8cf6a1f1ba8cc87b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 11:05:18.849120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Aug 09 11:05:18.867974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:05:18.868951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 11:05:18.869836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 11:05:18.870677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:05:18.871540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:05:18.874070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': 'a2ffaa74d754448a8cf6a1f1ba8cc87b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:18.876110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 11:05:18.878078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:18.886163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:18.886322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/477] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:18.886322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:18.982790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fe8b8-1696-4c79-99f5-7bc8c917139b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:19.019603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:19.021626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:19.021948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:19.022216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:19.022955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:19.024416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:19.050070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/478] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:18 2022] POST /identity/v3/users => generated 249 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:19.094699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7f4f6-a15c-4f16-8b39-58b7d676cb45 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:19.120999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:19.154035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:19.154607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:19.154881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:19.155148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:19.171277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b41fb4f1-b6ba-463b-8377-3de4efc5b656 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:19.176033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:19.176982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:19.177770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/479] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:19.183730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:19.302604 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de80054-3e59-4870-866d-38bde31a4c7a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:19.304476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:19.338489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:19.339104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:19.339382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:19.339651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:19.340411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:19.340747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:19.342142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8125f-f139-4154-ade1-ef807fae5a91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:19.350867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/480] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:20.617447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09d0b-1b80-4a2c-84dc-e455504e9d9b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:20.618011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09d0b-1b80-4a2c-84dc-e455504e9d9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:20.618102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f09d0b-1b80-4a2c-84dc-e455504e9d9b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:20.619888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/481] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:20.625165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4688fc-7e4a-4b02-8ef3-be650a5809ae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:20.625638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4688fc-7e4a-4b02-8ef3-be650a5809ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:20.625762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4688fc-7e4a-4b02-8ef3-be650a5809ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:20.650926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad4688fc-7e4a-4b02-8ef3-be650a5809ae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:20.652105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad4688fc-7e4a-4b02-8ef3-be650a5809ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:20.729893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/482] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:20.750744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7966d-dc86-4c99-bff9-a87e02bf5e1c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:20.751025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7966d-dc86-4c99-bff9-a87e02bf5e1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:20.751377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7966d-dc86-4c99-bff9-a87e02bf5e1c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:20.752992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/483] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:20.758156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:20.758468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:20.758708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:20.769500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7966d-dc86-4c99-bff9-a87e02bf5e1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:20.794329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:20.795517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:20.932298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7966d-dc86-4c99-bff9-a87e02bf5e1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:20.948418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/484] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:20.962804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:20.963428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:20.963702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:20.963965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:20.974925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:20.979881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:20.980756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:20.981566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:20.982152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/485] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:20.988828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:20.989430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:20.993034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.033877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.101450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc7fe8-27ae-483c-939e-cf6f823c91e6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.114838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e69c1-16a3-4cc3-94a1-10975cdf73e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.138290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.139266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.139744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.140204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.152020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0fa7004b-57cc-4d71-9af9-031fc898b341 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:21.157633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:21.158524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:21.159353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:21.159942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.159942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.159942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.159942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.160203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:21.160855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.161890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12859700-1b64-497d-89ba-8428e723a505 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.162696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/486] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:21.164384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e962df-60e0-4e34-b0bf-d65722b6fec8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.164870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e962df-60e0-4e34-b0bf-d65722b6fec8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.165325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e962df-60e0-4e34-b0bf-d65722b6fec8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.166712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/487] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:21.168590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e962df-60e0-4e34-b0bf-d65722b6fec8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.184559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/488] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:21.189225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.189225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.189225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.206478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:21.207681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:21.245237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.288290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/489] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:21.288290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.292522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e962df-60e0-4e34-b0bf-d65722b6fec8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.313500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.342294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.343032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.343212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.343488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.344156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:21.344549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.345990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.355167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/490] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:21.360541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.412836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cbdf3-2449-4e14-86ca-1bb74865c1a6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.459458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.461072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.461692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.462252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.466654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ecfa8-be08-4818-b5ca-25abd428c2b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.475652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.478217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.495079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:21.499933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:21.500921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:21.501753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:21.501753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:21.501753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:21.501909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/491] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:21.507307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.513061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.513542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.513817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.514132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.526004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:21.531142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:21.532343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:21.533148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/492] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:21.533788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.627674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.657121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8fc42-6090-4497-968e-d49512e4f03e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.658679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dce3d3-fe7a-469c-95ab-bf8614d4287a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.692537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.693558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.694043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.694522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.695562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:21.696067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.696501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.697231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.697606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.697972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.698239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.710063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.711605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.711990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/493] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:21.713171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:21.727462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:21.728437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:21.729289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:21.729289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:21.729289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:21.729289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/494] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:21.730390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.763385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.819164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08910312-578a-49b9-b687-dc597c10f41c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.851211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810a7d-f708-43e0-a4ba-83b3079de07d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.853055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.853660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.853929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.854192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.854852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:21.855146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.856546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.864701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/495] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:21.865494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.890020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:21.898287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:21.898886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.899194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.899430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.907241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:21.911296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cdb5e986-c0b9-40db-b1b0-c5c893fdcd8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:21.924002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:21.925075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:21.925979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/496] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:21.926546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776befed-9a97-4f9f-a481-bf1fb5821935 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:21.926546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776befed-9a97-4f9f-a481-bf1fb5821935 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:21.926799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776befed-9a97-4f9f-a481-bf1fb5821935 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:21.927999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/497] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:21.928700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776befed-9a97-4f9f-a481-bf1fb5821935 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:21.971971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb44cbb-cf71-4a0f-a88c-282e3c223b19 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:21.980461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.006064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.006668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.006950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.007221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.007858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:22.008156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.009564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e74b-b7e5-456a-a131-ffb11f43f155 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.021459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/498] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:22.022628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37b442-e109-43aa-b1b3-c22b2e6377f2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.022908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37b442-e109-43aa-b1b3-c22b2e6377f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.023179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37b442-e109-43aa-b1b3-c22b2e6377f2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.024671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/499] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:22.025380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37b442-e109-43aa-b1b3-c22b2e6377f2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.033862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776befed-9a97-4f9f-a481-bf1fb5821935 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.070772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.071384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.071649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.071912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.083042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.084518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ee222029-a960-4955-b113-4136aafa3b35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:22.098088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:22.099149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:22.100057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:22.100057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:22.100057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:22.100057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/500] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:22.100381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.100536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.100795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.118277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:22.119479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:22.127717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37b442-e109-43aa-b1b3-c22b2e6377f2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.154096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.162324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.162921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.163191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.163452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.164176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:22.164503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.165920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db104384-9995-4ca9-8845-40c1073c5765 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.186555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/501] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:22.187392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.220582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/502] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:22.221837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.222203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.222351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.240249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:22.241487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:22.262609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.293243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ef641-464f-4d0e-8153-1ff892206c96 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.307418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/503] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:22.308240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.331461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.332084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.332386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.332651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.339701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.341217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:22.352249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:22.353226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/504] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:22.354134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.385423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.416829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96d2b5-0466-47e4-b83e-b9577fa091e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.449242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.449850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.450133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.450398 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.451041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:22.451341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.452765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.452877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2cdf0-edff-4dd5-9b66-0f09476b0241 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.463774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/505] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:22.464655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.481820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.489179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.489797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.490062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.490325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.500226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:22.505279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:22.506209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:22.506990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/506] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:22.507625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.575734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a265ab-0144-41af-8683-52116e3eaa61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.609140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.610091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.610745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.611019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.611284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.618835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f9a5-8228-488d-a479-3474d0dc629b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.652220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.652844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.653143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.653414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.663686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.665173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.678249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:22.679125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:22.680068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:22.681066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:22.681066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:22.681066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:22.681233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.681300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/507] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:22.681300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.698372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.779415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a88a5-e1f4-4890-8cf4-f7daf882294a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.781275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/508] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:22.782610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.811642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.812256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.812618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.812925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.813862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:22.814186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.815619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.852023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/509] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:22.852907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.855644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.881759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366d335-a54a-4763-bff1-9d8fade2fd9b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:22.899641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:22.918449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:22.919077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:22.919352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:22.919615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:22.926980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:22.928451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:22.942322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:22.943241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/510] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:22.944165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:22.966259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a03e12-ef89-49af-b20f-243b9b1cc6c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.000842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.001487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.001761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.002014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.002679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:23.002995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.004399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.012088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/511] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:23.016906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.048484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc621a68-3d10-4e86-a1cd-0167cabc07fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.074365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.082590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.083193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.083472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.083742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.084401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:23.084705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.086087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.098832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/512] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:23.099651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.116793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197d971-b2fe-43f4-973e-100ddbba0eaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.138147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.153867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.154492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.154765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.155006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.165838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.167343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:23.181422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:23.182467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:23.183471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:23.183471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.183471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:23.183471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/513] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:23.183729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.210404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b005ee-77f7-461e-a292-0cf4dcf6565d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.249440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.250042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.250307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.250613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.286388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2db01c-73da-4084-bbfe-47aabef4ab98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.302572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.320626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.321225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.321498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.321585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.321763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.322478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:23.322777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.322952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.324198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.341266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/514] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:23.341882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.356019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.427065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/515] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:23.427883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.442652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1860a9-7cf3-4f6c-9674-d4f0b9f781bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.476654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.477263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.477533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.477799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.488546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.490025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:23.504613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:23.505641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:23.506640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:23.506640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.506640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:23.506953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/516] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:23.507873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.526237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb35831-c01f-48a0-a500-c444bebb880e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.534732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.563636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.564127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.564428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.564692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.565325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.566745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.575852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.588578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/517] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:23 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:23.589394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.622559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4672d-f54e-450b-9d22-3870bd0e6417 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.663478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.664850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.665125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.665391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.666101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:23.666398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.667817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.681291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/518] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:23.682221 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.710822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4560db0-cf36-42f9-b39f-ff76ddea0f45 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.748098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.748844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.749118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.749472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.784294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.802455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2930e92-98a4-4553-b1cd-382e3d0206b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:23.822873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:23.834451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.835968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.838930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:23.839559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:23.839829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:23.840091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:23.850203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:23.851663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:23.867903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:23.868841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:23.869679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:23.869679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:23.869679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:23.869679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/519] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:23.869883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.946283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/520] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:23.946934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:23.983643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309498c-933b-4fb3-b235-4ac5e9fcb300 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.020655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.021576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.021847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.022112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.022745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.023038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.024432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.031963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.037311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/521] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.041140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.047504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.057054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc0bf-e8c1-4672-9aa1-6188812f5cd4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.091456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.092000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.092299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.092559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.099278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.100756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:24.112314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:24.113168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:24.114019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:24.116621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/522] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:24.117823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.151185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b48e1-fd07-4813-8075-f87e22672b1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.187281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.187910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.188193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.188623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.189267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.189565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.190941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.210035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/523] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.210035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.230856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77820958-f755-45a3-8ae0-ae281f53c5a2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.265675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.266365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.266640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.266981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.275338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.278293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.279752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:24.293748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:24.294856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:24.295856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:24.295856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:24.295856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:24.295856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/524] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:24.296088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.299794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.326969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8486499-5311-45b7-b264-b9e5c636cf7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.364158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.364726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.364983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.365253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.373917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.375851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:24.390194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:24.391219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:24.392081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:24.398236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/525] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:24.398236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.401007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdddf-a1af-4a0b-9917-83f63545aab4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.435115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.435852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.436233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.436614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.437383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.437774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.439368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.456845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/526] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.457706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.508516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf18ad-3431-4815-9ec8-ee9b468d4f64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.526541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.526948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.546083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.546827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.547226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.547496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.554320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.555756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.566114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ff050-b8df-4ce4-b158-d27f5452efd2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:24.568234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:24.569094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:24.569938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/527] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:24.571592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.608669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.609540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.610013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.610414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.611374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.611822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.614000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.637205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/528] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.637205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.679149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba788e79-8171-422d-adff-9755747009b3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.716448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.717059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.717338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.717605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.718335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.718638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.720209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.744060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/529] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.744060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.756712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375ca9-c046-4ca3-85d5-63cc9b8497bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.779597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.784823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:24.798437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.799052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.799327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.799593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.800349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.800654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.802094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.807767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/530] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.808557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:24.877058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74285add-85ae-4650-b6e6-72c17125873a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.935980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:24.936905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:24.937222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:24.937607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:24.938899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:24.939210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:24.941719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:24.954026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a666c-d822-4619-b9a4-21a3c0f9e3a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:24.967394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/531] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:24.968713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.002896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.003527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.003810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.004151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.011210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.012687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:25.024766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:25.025799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:25.026625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:25.030442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/532] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:25.031361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.083600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.089481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.096984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b2da1-afb7-484f-a5d6-fc1de65eb085 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.137210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4227c3-da24-4b12-870c-6c3d64e2356c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.141014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.141683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.141954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.142254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.180135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.180732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.181371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.181646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.182677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=befb4776ff7b4014997d32e4697f29a6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:25.182984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.184474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.192208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/533] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/role_assignments?user.id=befb4776ff7b4014997d32e4697f29a6&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:25.197206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.255675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.257574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.395101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.409649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/534] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:25.410888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.415374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3c838-ca7c-4702-b065-4e52e12c07dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.439264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.479743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.480417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.480638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.480904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.481600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:25.481896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.483584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.513084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/535] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:25.520311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.552570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296151-d17e-4cff-a12d-e16c23c16188 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.611882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.612573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.612845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.613097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.625337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.627003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:25.642810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:25.643752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:25.644609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:25.644609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:25.644609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:25.644609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/536] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:25.649999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.705459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.720135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58f77a-b645-4c8c-af6f-72bf52c38817 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.763210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bcd78-76d4-45b9-a083-838db2ee995e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:25.765353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.766050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.766382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.766712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.768438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=32cbfe0d7b584729b45b85c65377648f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:25.768811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.770332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b909b9-1895-4ed5-a4a0-09a44210204c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.780351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/537] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/role_assignments?user.id=32cbfe0d7b584729b45b85c65377648f&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:25.782045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316e41-ca90-4afd-a2d8-6a679b3b845d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.782501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316e41-ca90-4afd-a2d8-6a679b3b845d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.782942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316e41-ca90-4afd-a2d8-6a679b3b845d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.784479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/538] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:25.790305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.791030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.792686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.812637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:25.814735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:25.829300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:25.830049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:25.830323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:25.830590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:25.831247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:25.831541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:25.833104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:25.851310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:25.863958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/539] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:25.869995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:25.930989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/540] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:25.954363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.016689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c809-b553-44f3-8c52-4ed89d4e0e78 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.027167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.054205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.054929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.055212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.055482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.062786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.064452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:26.076456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:26.077328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:26.078137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/541] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:26.078646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b8a11-e92d-432e-a69d-65b759d08448 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.085271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.113280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.113956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.114228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.114490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.125092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:26.130003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:26.130869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:26.131663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/542] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:26.137159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.177877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.281231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f225656-fb5c-4ab6-9076-1494edda5162 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.303776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b1e04-5da3-4251-953d-8a25db67306c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.358449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.359052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.359323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.359585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.360587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:26.360901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.362444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.363781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.364651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.365149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.365594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.366467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:26.366961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.368670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.374927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/543] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:26.380981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.393150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/544] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:26.400029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.454468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.505668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.529743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb993b45-af12-4948-acd4-57d6fd51acef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.568577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb373fc-9c70-43a5-a4ae-06ca6cd8dfdb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.591462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.592467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.592955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.593406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.605038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.606903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:26.623423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:26.624431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:26.625373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:26.625373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:26.625373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:26.627852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/545] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:26.634902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.642734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.643342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.643613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.643875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.645352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f3216e5d1c354d8e8d5c1fbf449f882c) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:26.645352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.646676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.652439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/546] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/role_assignments?user.id=f3216e5d1c354d8e8d5c1fbf449f882c&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:26.771373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314549be-d97b-41d1-81a1-df8bf0d3873d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:26.815786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11583994-c606-4f26-b46e-6642733caacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:26.816431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583994-c606-4f26-b46e-6642733caacd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.816705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583994-c606-4f26-b46e-6642733caacd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.816967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583994-c606-4f26-b46e-6642733caacd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.817610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583994-c606-4f26-b46e-6642733caacd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:26.817902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583994-c606-4f26-b46e-6642733caacd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:26.819308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583994-c606-4f26-b46e-6642733caacd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:26.829653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.839999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/547] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:26.846015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:26.991501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:26.993217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcd9c0-9cae-4d32-8d2c-799a10020894 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:26.993657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcd9c0-9cae-4d32-8d2c-799a10020894 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:26.994053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcd9c0-9cae-4d32-8d2c-799a10020894 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:26.995487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/548] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:27.000878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.001203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.001472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.009937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc79e8-922b-4820-b382-7c5f4888de4c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.031744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:27.035498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:27.065070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.066100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.066565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.067114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.080914 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.083895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:27.102051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:27.103474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:27.104621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:27.111933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/549] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:27.118122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.185664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/550] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:27.229962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.232005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5807f-e26b-4150-8d50-93f2219a450d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.271889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.277302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.296943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/551] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:27.304645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.322532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:27.417951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb044a2-4b88-411e-8429-7a5c1dcc38c5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.459701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7744c84-9525-4207-8d77-a404e7e65047 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.474574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:27.486234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.487112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.487730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.488843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.508438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:27.516085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:27.516994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:27.517978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:27.521207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/552] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:27.529222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.544880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.545488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.545757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.546017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.672793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.674278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.688694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:27.723148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c934285-3332-4a74-b0f3-a7faa2d029be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.759773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.760804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.761372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.761844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.762730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:27.763215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.764898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586ef0b-44cc-4c8d-b072-668f9d21c9d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.774807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/553] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:27.776805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f29a-9fef-475e-ab17-6d25a3b8b2a1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.777292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f29a-9fef-475e-ab17-6d25a3b8b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.777758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1f29a-9fef-475e-ab17-6d25a3b8b2a1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.778171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/554] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 11:05:27.783526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.783983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/555] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:27.785057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1f29a-9fef-475e-ab17-6d25a3b8b2a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.844737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:27.884345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f6804-27c5-46da-aa28-409a945cadda None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.891728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1f29a-9fef-475e-ab17-6d25a3b8b2a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:27.895170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:27.929037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.929697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.929974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.930308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.931560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:27.932971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.934314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.934971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.945832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.948456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.950192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:27.952108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:27.965905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:27.966846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:27.967817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:27.967817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:27.967817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a870c581-d8f3-4e1d-a66b-d40fe6770626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:27.968498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:27.969363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:27.970677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:27.970677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:27.970677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:27.970677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/556] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:27.970901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:27.971347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:27.971347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:27.973064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/557] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:27.980453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:27.997756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:27.999254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:28.070259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/558] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:28.071777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.093132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e0ffb-251b-41ca-ac76-648203e659bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.128086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.129705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.130478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.130880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.131213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.131925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.132304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.133748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.147691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/559] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:28.152883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.180672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.205724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1157a5-a73e-49d7-a967-0481f6115fb6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.242750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.243348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.243623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.243883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.244546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.244838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.246215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.257503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/560] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:28.258285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.267626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063ed3a-8d6f-42d5-ae4c-fd28b0f13f82 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.304489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.305375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.305856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.306397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.317303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-83636675-9acf-4b2e-b00c-8bff23d6e81e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:28.322916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:28.323810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:28.324684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:28.327327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/561] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:28.328675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b50c-6994-41e8-809f-476de03d7f7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.328947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b50c-6994-41e8-809f-476de03d7f7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.329214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b50c-6994-41e8-809f-476de03d7f7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.330550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/562] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:28.331093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b50c-6994-41e8-809f-476de03d7f7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.366226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.375105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a83e-0c1c-47d5-bcc4-a75bd2683743 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.411158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.411785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.412059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.412428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.419328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.420929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.423398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ef89d385-e241-4bea-a4ba-3b66aa577572 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:28.433309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:28.434338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:28.435376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/563] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:28.435860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.436050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.436675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.450772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b50c-6994-41e8-809f-476de03d7f7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.454929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:28.456146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:28.488744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.489371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.489645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.489968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.498735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.500300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:28.515073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:28.516356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/564] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:28.522486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.536760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/565] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:28.536760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.614779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.633265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75596-5f33-4fd5-8504-9e64e8997656 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.637313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148439d1-2e30-43e0-aab4-972b6e7a5b21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.668248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.668991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.669329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.669706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.670424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.670781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.671781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.672473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.672589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.672906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.673249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.673992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.674354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.675814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.689684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/566] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:28.689684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.690504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/567] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:28.691422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.694303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.796889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f87b1-5234-4691-9fee-658a9d683776 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.812807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461e83-2061-4f8b-8bf9-f72119c3a3bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.835397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.836347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.836658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.837019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.847842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:28.850588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.851198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.851472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.851736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.852507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.852790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:28.853532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:28.854402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:28.855222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:28.855842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:28.855911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/568] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:28.855911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.860115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.877989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/569] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:28.883118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:28.960829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:28.963664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596c9d3-57e9-4338-8381-7d3eb7ae882c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:28.997583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:28.998179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:28.998455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:28.998721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:28.999438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f55820319ba442aa8288cf179681b2d9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:28.999741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.000888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227dcd9-4ed9-41cd-85fe-2effc13cd3c1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.001260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.006459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/570] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/role_assignments?user.id=f55820319ba442aa8288cf179681b2d9&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.007293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.037665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.038305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.038578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.038844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.050288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.052598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:29.070007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.071049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:29.072031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:29.072031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.072031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:29.072031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/571] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.072031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.101662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.138861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0f1d7-3bc5-402a-ace6-4272fdf8c67b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.176197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.176855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.177131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.177472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.178053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:29.178351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.179780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2e8be-d29c-4937-b7b2-ed2ac9cda440 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.192189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/572] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.192403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe4026-7e76-4b62-85ec-b32d9627d291 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.192514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc6a25-7dd5-451a-ad2d-b38281fe72bc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.192514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc6a25-7dd5-451a-ad2d-b38281fe72bc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.192514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc6a25-7dd5-451a-ad2d-b38281fe72bc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.192514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/573] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:29.192514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc6a25-7dd5-451a-ad2d-b38281fe72bc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.209060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.231506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.232051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.232346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.232613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.243456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.245048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:29.259365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.260503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:29.261356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:29.261356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.261356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:29.261356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/574] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.261356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.305697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc6a25-7dd5-451a-ad2d-b38281fe72bc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.335774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.344181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.344822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.345088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.345352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.345985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:29.346274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.348052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38723e2e-8a64-4e62-a1f2-4c554244ca0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.365063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/575] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.365063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.365063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.365063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.378724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ddcc3-c802-4d00-be8d-46e3312a3eb5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.381247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:29.382410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:29.413955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.414572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.414843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.415097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.425762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.427206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:29.441630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.442609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:29.443474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:29.443474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:29.443474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:29.443474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/576] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.443691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.458909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/577] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:29.459778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.479015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.566034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852d422-1521-490b-aa1d-4161180418a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.566929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6725-07fa-4518-9059-42a5f8c8fbf3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.588260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.603246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.603870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.604003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.604186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.604491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.604608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.604883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.605303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.605942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:29.606344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.607792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.611763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.613258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.625710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/578] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.625710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:29.626503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.627412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/579] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.628393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.709009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.733512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b579bf-0123-4c63-b7bc-85d03e15d541 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.744820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08004045-8aa0-42fe-84c0-febb8de5ae24 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.771239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.771975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.772466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.772749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.773500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:29.773903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.775358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.779358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.779974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.780254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.780547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.791708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/580] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.791708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.796602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:29.801937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.808160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/581] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.809270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.851625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.901407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203eeb1-c722-42ab-8c5a-d0bef8984936 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.916529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9b834-1f46-434e-aa51-d1a7ac081448 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:29.940870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.941544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.941819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.942083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.949584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.951274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.953024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:29.956600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:29.957312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:29.957675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:29.958180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:29.959114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:29.959502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:29.961046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:29.963336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:29.964684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:29.965781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:29.974107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/582] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:29.974107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:29.983702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/583] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:29.984670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.080225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628f9b4-2f82-4ae4-960b-98295735f9aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.087316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.091392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b7ff9-31f0-4bde-a7d9-6c225f8cf3ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.119557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.120201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.120570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.120905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.129846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.130002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.130496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.130765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.131026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.131299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.131710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:30.131967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.133326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.141745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/584] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:30.143065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:30.143980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/585] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:30.145518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.151519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.213945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.270363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e0a2-2988-4a86-8ec5-fd65ae78909d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.280591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c743-3c76-41c1-b904-c88a8d023764 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.307732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.308389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.308659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.308919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.322041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.323505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.324371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.325359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.325834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.326469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.327248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:30.327638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.329338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:30.338681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:30.339833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:30.340928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:30.340928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.340928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:30.340928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/586] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:30.340928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.348962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/587] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:30.349959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.373841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.484186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced1f44-2edb-460a-b122-924dd23fe94c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.484420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072daa4-6c18-430e-9bca-538891d40cf0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.497499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.522993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.523848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.524180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.524728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.525562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:30.526042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.527763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.534506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.535587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.536019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.536532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.552964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.554847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.558492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/588] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:30.558492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:30.575304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:30.576583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:30.577668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:30.577668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.577668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:30.578861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/589] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:30.583824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.659706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.665489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a18020-7c7a-4186-bb55-2301439f39f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.690539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813caab-c5c1-4539-96ac-aa81887dd49f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.701036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.701719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.701998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.702265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.702940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:30.703238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.704847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.720589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/590] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:30.720589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.732240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.732875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.733150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.733419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.749861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=fbbd789ff1f244eb98d2e5dd001a81c1, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.751389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.752228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.765057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/591] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:30 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:30.766267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.865523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791edd1-df5c-4169-9f7e-5e369e6106cd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.909975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.910539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.910818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.911082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.919855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f10df-7ee5-4b6a-ac76-c2e77d60faec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:30.926013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.927509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:30.946655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:30.947678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:30.948606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:30.948606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:30.948606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:30.952722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/592] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:30.953930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:30.956027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:30.974332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:30.975054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:30.975334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:30.975710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:30.976423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:30.976725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:30.978086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:30.992546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/593] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:30.994294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.051427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.071752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ef3a4-d242-4faa-9f78-e41e0217a7ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.103351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a3132-567a-4074-9777-efe9d89561e5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.106543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.107170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.107443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.107710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.114656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.116120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ecfa256d-b6be-488d-8d2e-879f10147384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:31.127558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:31.128520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:31.129402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/594] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:31.130330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc211a-9173-4d68-ab83-4dc37f481f7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.130330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc211a-9173-4d68-ab83-4dc37f481f7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.130478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc211a-9173-4d68-ab83-4dc37f481f7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.135928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/595] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:31.135928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc211a-9173-4d68-ab83-4dc37f481f7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.151605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.152330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.152680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.153014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.153734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:31.154032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.155420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.172468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/596] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:31.172468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.215033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.254853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc211a-9173-4d68-ab83-4dc37f481f7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.277314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13432-0be3-479a-b2c4-6905f3a7ca3a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.291237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.291867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.292148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.292441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.300202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.301672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.304568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b076182c-8824-4b6e-b975-e39561fba020 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:31.313787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:31.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:31.315806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/597] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:31.316414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.316414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.316640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.322784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.323572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.324055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.324378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.325057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:31.325353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.326769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.335316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:31.336530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:31.343826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/598] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:31.343826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.412344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/599] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:31.412344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.459954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acae30-bcb7-4889-b945-e1e8b5b0f8fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.462654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.505819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.506582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.506946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.507305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.514173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.515799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.525070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49687363-f90a-4a80-bbed-bc224ead6f5a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:31.527865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:31.528940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:31.530009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:31.531102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/600] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:31.532606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.562357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.569778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.570385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.570655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.570922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.571650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:31.571944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.573406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.591307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/601] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:31.592180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.646834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a1939-2bd4-4f8d-a3bc-5b7da9576e35 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.684812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.685460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.685735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.686003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/befb4776ff7b4014997d32e4697f29a6/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.701247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=befb4776ff7b4014997d32e4697f29a6, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.701447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec420-3600-4e91-9e3e-4ea579dd16cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.702723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.716155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/602] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:31 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/befb4776ff7b4014997d32e4697f29a6/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:31.717195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.750404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.755517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.756469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.756841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.757393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.771279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:31.776492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:31.777681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:31.778687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:31.785695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/603] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:31.785695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.853201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:31.869731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc92d6-41d9-4423-a578-9b69923e2d7a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.920009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.920707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5807-307e-427b-ab6d-2c0e6ae7488b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:31.921447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.921861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.922317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.923297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:31.924593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.926173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.941807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/604] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:31.942919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:31.962383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:31.963494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:31.963830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:31.964159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:31.978446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:31.980909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:31.999133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:32.000226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:32.001166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:32.001166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:32.001166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:32.001166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/605] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:32.001423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.033461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.067781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc42cd-a5d7-4e71-a36f-f53a444450b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.104951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.105303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.105585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.105853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.106509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:32.106806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.108261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.122634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/606] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:32.122634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.136879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.140966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3d68-f7da-44c0-bfa1-eb8006358c29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.178951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.179822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.180299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.180682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/32cbfe0d7b584729b45b85c65377648f/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.196768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=32cbfe0d7b584729b45b85c65377648f, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.198407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.216358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/607] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:32 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/32cbfe0d7b584729b45b85c65377648f/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:32.217516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.270203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca48d2-2025-4d5f-ad5d-b41929e3bcc1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.339775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060712-4520-438a-b146-715b24ab809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.340419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060712-4520-438a-b146-715b24ab809f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.340690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060712-4520-438a-b146-715b24ab809f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.340953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060712-4520-438a-b146-715b24ab809f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.342407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1060712-4520-438a-b146-715b24ab809f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:32.342702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1060712-4520-438a-b146-715b24ab809f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.345861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1060712-4520-438a-b146-715b24ab809f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.347783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.370794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/608] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:32.377130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060712-4520-438a-b146-715b24ab809f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.398486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b327aad-0589-4286-b866-42e52de56e09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.439568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.440202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.440514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.440783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.453216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.455202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:32.474879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:32.475876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:32.476847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:32.476847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:32.476847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:32.480340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/609] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:32.484386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.487675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.543968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060712-4520-438a-b146-715b24ab809f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.585841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.586776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.587186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.587602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.595118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.596950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.599297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f5ae9-768d-4c55-9860-2122754710b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:32.609624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:32.610708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:32.611625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:32.613723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/610] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:32.619137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.660211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.661129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.661535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.661932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.662728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:32.663174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.665929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.692261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/611] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:32.699334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.722479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.797663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:32.821312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce048a2-c501-476e-8834-978085ad3313 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.872954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809af4f-efd0-44a5-8287-1f14aa19f72b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:32.888691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.889294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.889615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.889879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.890700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:32.891177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.892867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.918259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/612] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:32.923175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:32.930831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:32.931723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.932217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.932798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.941381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:32.943190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b24041e8-3ebe-45ac-8b8c-e6c11cfad1bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:32.956097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:32.957230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:32.958158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:32.960842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/613] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:32.962282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960e87-45bf-4bcc-bd0c-f1815b18b9ce None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:32.962783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960e87-45bf-4bcc-bd0c-f1815b18b9ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:32.963241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81960e87-45bf-4bcc-bd0c-f1815b18b9ce None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:32.968447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/614] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:32.970130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960e87-45bf-4bcc-bd0c-f1815b18b9ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.031207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17a635-e960-4e90-8b2a-ff52b41083f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.046001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.068251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.069144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.069277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.069441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f3216e5d1c354d8e8d5c1fbf449f882c/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.083136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81960e87-45bf-4bcc-bd0c-f1815b18b9ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.085359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=f3216e5d1c354d8e8d5c1fbf449f882c, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.086873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45e624-e457-417e-8f91-691f45be606d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.108594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/615] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:32 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f3216e5d1c354d8e8d5c1fbf449f882c/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:33.109733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.111628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.112081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.112509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.125193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.126028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.126919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.127166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.129053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:33.129053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.130270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.152339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/616] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:33.158411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.159215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:33.162252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:33.300967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/617] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:33.332122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.337770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b743e0-33d9-4faf-9657-21d384e19142 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.374916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.375591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.376085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.376169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.387198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.388895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:33.415316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:33.416313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:33.417176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:33.417176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:33.417176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:33.417176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/618] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:33.422174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.440509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.450195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7603d-640d-495d-9c45-e33ba0952b4b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.463854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.521316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.521952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.522224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.522487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.536512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fd02e-6403-4a2b-9b8c-79d30ea73f7b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.541426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:33.546021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:33.547273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:33.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:33.556632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/619] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:33.557561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.575259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.575870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.576145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.576488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.579499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:33.579809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.583502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.613379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/620] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:33.619228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.727065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.736890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e9f85-24d2-48fc-aafa-9a74effc4780 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.774224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.774850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.775121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.775385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.778820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0b230-7a62-456d-a51e-a2a0f45af938 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.783481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.784287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:33.795986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:33.796912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:33.798094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/621] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:33.806342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.827526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.828859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.829278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.829701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.830491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:33.830926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.832553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:33.841117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/622] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:33.846804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:33.880619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:33.943528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e366-7cdb-4dc4-ad02-b97015074bc2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:33.985430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:33.986412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:33.986915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:33.987376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:33.988390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:33.989030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:33.990670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc93fd-4240-4e16-bb02-1e852187d9a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.004730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/623] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:34.006261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a184-8b8d-42b5-b3fd-64f6be4bd153 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.006786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a184-8b8d-42b5-b3fd-64f6be4bd153 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.007269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a184-8b8d-42b5-b3fd-64f6be4bd153 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.012800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/624] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:34.014194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.014670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.015222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.034907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:34.036533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:34.041391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588061a-c2f4-4d26-bf8c-1b587930f788 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.045432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.079787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.080814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.081267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.081740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.094082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.095775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:34.111116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:34.112401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:34.113329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:34.113329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:34.113329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:34.115166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/626] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:34.116288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.116659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/626] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:34.120307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.193225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.235886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef3fb-b7d4-4e3b-91c0-a042f7ae6c99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.253748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c963d-7599-4519-afb3-eef364bc08ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.279346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.280012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.280298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.280579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f55820319ba442aa8288cf179681b2d9/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.299437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=f55820319ba442aa8288cf179681b2d9, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.300453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.300912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.301583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.302055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.302516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.303607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:34.304666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.306424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.324340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/627] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:34 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f55820319ba442aa8288cf179681b2d9/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:34.324340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.326150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/628] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:34.332084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.339178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.513185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505ce2-c42f-4531-aa8d-8f32b7144a26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.525915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc39c1d-9d51-4fd3-a3a6-db4ca0896b03 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.538770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.566592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.567216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.567484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.567802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.578396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.579280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.579726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.580246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.584919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:34.589585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:34.590536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:34.591793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/629] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:34.593582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.595461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.597053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:34.609291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:34.610377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:34.611319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:34.614458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/630] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:34.621303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.703422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.727839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c6a56-dcdc-4aaa-a115-501d49faaa1b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.746662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdcd4a-3529-467d-bd24-928da887c26f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.762165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.762774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.763043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.763361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.763962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:34.764255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.765845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c86bd5-83b3-4d7e-87b4-2e2204cbfec4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.774547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/631] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:34.775829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3414271-b996-420d-8942-60095aa92ace None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.776106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3414271-b996-420d-8942-60095aa92ace None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.776411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3414271-b996-420d-8942-60095aa92ace None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.781726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/632] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:34.781726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3414271-b996-420d-8942-60095aa92ace None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.794541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.795548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.796021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.796535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.797504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:34.798058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.799789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960949e-0c4c-434a-8e1b-6276ca6af6d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.819437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/633] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:34.821125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.821659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.822143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.826900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:34.843612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:34.845455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:34.906998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3414271-b996-420d-8942-60095aa92ace None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:34.934651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/634] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:34.936610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:34.955588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:34.956218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:34.956518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:34.956782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:34.968034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:34.970320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:34.985993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:34.986940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:34.987810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:34.987810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:34.987810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:34.987810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/635] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:34.988578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.008969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.072241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a4249-30f3-46f1-a8cd-a9fb2412a56a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.099002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.109318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.110528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.110863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.111137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.112644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81e00-7060-416f-8263-ba4f79ba9192 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.158516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.159146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.159417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.159679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.171711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:35.177633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:35.178603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:35.179426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/636] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:35.180098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.200130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.201652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.256876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.311788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9cfb7-fac1-48ca-b063-6dab500495e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.316977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/637] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:35.316977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.353694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.354212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.354484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.355071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.355626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:35.355928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.357383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.370732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/638] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:35.371557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.387935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.444212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae7a5f-6e29-4cba-876b-7716df75243c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.483628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee12f8-1ec0-43b2-aae0-3e5357443884 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.486522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.487128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.487394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.487649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.488328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:35.488621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.489993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.498868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/639] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:35.499677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.515085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.523988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.524647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.524917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.525194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.536678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.538332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1389a0d0-b166-49cd-9fa6-8b30635f8fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:35.555297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:35.556440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/640] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13380-ab50-47df-8106-afc38b754d8b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13380-ab50-47df-8106-afc38b754d8b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.562459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13380-ab50-47df-8106-afc38b754d8b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.563516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/641] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:35.564522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee13380-ab50-47df-8106-afc38b754d8b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.614542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdf396-add6-4089-8ab8-883d024d4d62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.631946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.654392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.655404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.655682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.655951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.664013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.665515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.675439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee13380-ab50-47df-8106-afc38b754d8b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:35.677002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:35.677979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/642] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:35.678917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.714930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.715556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.715831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.716098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.727365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.728825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1219c072-7799-434d-8f4a-227348190c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:35.742412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:35.743499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:35.744588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:35.744588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:35.744588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:35.744588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/643] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:35.744984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.745261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.745528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.760330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.764480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:35.765688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:35.800691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6bf6f-b016-4670-bd33-585894417ae8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.835560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/644] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:35.836065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.836414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.837143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.837470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.837815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.838520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:35.838873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.840330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:35.856046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/645] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:35.861469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:35.891895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:35.950972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7842473-5a17-4724-868f-72d38bcb01b5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.978938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f22dd9-9f32-49d6-8587-75f1466f9b6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:35.986534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:35.987319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:35.987748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:35.988159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:35.989058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:35.989492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:35.991066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.009300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/646] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.010001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.015003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.021191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.022060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.022330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.022593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.023232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:36.023517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.024992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.040865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/647] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.040865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.118739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb609-b95a-4d0d-bebc-850b023da441 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.121143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.144732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e811-172d-4553-85b1-cd98057d8629 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.155399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.156319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.156730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.157352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.164329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.165924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:36.177036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:36.178073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:36.179090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:36.179933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.180590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.180867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.181132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.181380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/648] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:36.182266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.196232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:36.201472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:36.202367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/649] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:36.203358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.264578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.293840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b07-eefa-4a70-81b9-dca569e17f8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.315933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf307-29fd-4ec5-8a48-de876eac1d8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.327416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.328015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.328357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.328622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.352459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.353036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.353304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.353736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.359642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.361713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.363265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:36.375739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:36.376733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/650] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:36.377639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.407893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.409775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.483900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bdbba-1ba8-4365-aaa2-c0c8ab6f57d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.496245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.507179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/651] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:36.507932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.517589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.518187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.518457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.518726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.519410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:36.519708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.521482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.533526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/652] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.534307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.568349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.605633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbd9b0-c811-48ca-aa87-15ec5bbe7db9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.635128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79f39b-746c-4f06-8bf5-e0da28f320a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.638317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.638918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.639190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.639456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.640109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:36.640439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.642054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32814361-ffc8-4baf-962e-79dd7ddd95b3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.650040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/653] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.651188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20488e6-3d3d-4933-a837-282c00b2db40 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.651465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20488e6-3d3d-4933-a837-282c00b2db40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.651730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20488e6-3d3d-4933-a837-282c00b2db40 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.652920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/654] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:36.653594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20488e6-3d3d-4933-a837-282c00b2db40 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.675063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.675596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.675864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.676125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.676882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:36.677178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.678600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.696336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/655] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.697159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.726319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.756587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20488e6-3d3d-4933-a837-282c00b2db40 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.782239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.791575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.792200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.792493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.792762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.794896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1ec1f-51c2-47b7-b5b9-cdf69f318012 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.803552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.805043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:36.819086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:36.820107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:36.821021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:36.821021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:36.821021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:36.821021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/656] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:36.821448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.833090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:36.833674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.833943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.834205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.834921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:36.835219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:36.836673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6348c7a-babf-4d3b-b04f-5929f131a429 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:36.842160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/657] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:36.843247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:36.843522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:36.843845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:36.866241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:36.867808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:36.949704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/658] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:36.950158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce6d54-0e10-4318-a705-82fab8837dd4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:36.950505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:36.992228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:36.999702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.000545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.000818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.001081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.013520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.014959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:37.029477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:37.030444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:37.031463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:37.031463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:37.031463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:37.031463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/659] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:37.031463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.072434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.091047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29c9a6-1f76-4747-978a-c8ad70743353 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.127097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.127702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.128018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.132231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.151123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97c20b-2799-4d18-bd8c-62f31fe588d2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.186095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.186729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.186999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.187258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.188564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:37.188866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.190223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.206495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/660] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:37.206495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.211526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.213010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.240989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.305122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.315561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd5ce-2960-4813-acc8-7319e9b99333 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.330213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/661] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:37.330213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.357440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.358081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.358358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.358626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.370333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:37.375831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:37.376933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:37.377840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:37.380420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/662] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:37.381640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.438977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b01c6-5a6e-4a7f-875e-e55c249941d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.464333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.476682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.477283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.477556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.477818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.478443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:37.478734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.480092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.482841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d541e3-4272-4d5a-a7eb-9887b628dc13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.495703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/663] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:37.495703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.523037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.523660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.523935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.524194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.531347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.532902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:37.544148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:37.545062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:37.545947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:37.551692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/664] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:37.551692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.564725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.613703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d808043-772e-42e4-8e8e-23fb398f03b3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.649302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.649924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.650193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.650452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.661262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.662695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.671593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0bd5-57f0-4c5c-abc2-d58e3db97799 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:37.679345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:37.680319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:37.686105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:37.686105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:37.686105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:37.686105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/665] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:37.686105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.706459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.707053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.707324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.707588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.708244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:37.708802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.709904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.717772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/666] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:37.722471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.727684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.795298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.798072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8667f42-393b-43db-99a3-93ac2c3dc47a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.825125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ddd10-b42a-43af-b51a-aac1744a5738 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:37.834030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.834654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.835036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.835302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.843239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.844866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:37.857078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:37.857982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 11:05:37.858912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:37.859545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:37.859545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:37.859545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:37.859545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:37.859995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:37.859995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:37.861230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:37.862827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/667] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:37.863661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.873597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/668] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:37.874392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:37.973868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:37.980851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac95e3-6e48-4382-b5be-15f15fe03b7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.005134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65460a86-cdb1-487a-a15d-99e66827a828 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.017926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.018554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.018832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.019151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.019751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.020048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.021813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.034912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/669] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:38.035737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.043979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.049916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.050632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.050911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.051180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.061630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.063480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:38.079042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:38.080073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:38.081433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:38.081433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:38.081433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:38.081433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/670] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:38.081433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.151393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d75d0-54b6-452b-a4b2-5f603370a5c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.186593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.187198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.187500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.187777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.188523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.188902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.190237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.197609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722dc4-6c6f-46d9-b61b-fb9048ebdd5d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.206941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/671] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:38.207725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.226799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.237401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.238003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.238278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.238557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.239357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=befb4776ff7b4014997d32e4697f29a6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.239666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.241140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.246805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/672] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/role_assignments?user.id=befb4776ff7b4014997d32e4697f29a6&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:38.247568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.312227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.345311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18059b0b-ca10-4b31-a3a1-1525003f2c3a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.371936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f5abe-1f75-4e28-b2ba-841a0b616115 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.390474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.392171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.392669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.393152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.404311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.407227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.410717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.411414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.411832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.412167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.412930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.413294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.414804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:38.422608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:38.423574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:38.424494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:38.428427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/673] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:38.429620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.440779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/674] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:38.441774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.518952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.549154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfe0a1-b267-450c-9ca0-151232f370b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.563890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587f0a-147b-428c-9147-98b42df2e309 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.577004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.586462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.587075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.587399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.587645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.621752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.622825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.623537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.624340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.625518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.626123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.629839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.650354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/675] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:38.651974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.670123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.672094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.769189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fbf1a-a752-4ffd-9868-40bdc578eb57 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.769716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.804988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/676] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:38.806072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.806706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.807536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.807996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.808729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.819979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.821977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:38.837685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:38.838721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:38.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:38.846583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/677] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:38.846583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:38.879070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:38.948064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f74ec0-fa33-493a-a001-944358cba7be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.964029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b709123-9241-46d6-97b5-7a5e987124e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:38.986645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:38.987699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:38.987979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:38.988250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:38.989147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=32cbfe0d7b584729b45b85c65377648f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:38.989450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:38.990889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:38.997760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/678] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/role_assignments?user.id=32cbfe0d7b584729b45b85c65377648f&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:39.002056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.002697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.002970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.003236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.003893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.015949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.017542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ce688c5b-e0c4-4966-96f8-5c9b6a1446f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:39.035569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:39.036620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:39.037510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:39.037510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:39.037510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:39.041904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/679] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:39.042969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd911-5673-4d75-b1aa-85560839b94f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.043262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd911-5673-4d75-b1aa-85560839b94f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.043539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd911-5673-4d75-b1aa-85560839b94f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.049154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/680] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:39.049154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bd911-5673-4d75-b1aa-85560839b94f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.072911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.148648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.164642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2a988-ecfa-4cab-a607-5b5ba0bc0d70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.175810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bd911-5673-4d75-b1aa-85560839b94f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.210593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.211568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.212069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.212553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.213492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:39.213985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.215601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.216968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.217572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.217846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.218113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.218906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:39.219842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.221366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a4354-a7a0-4596-b77b-6cf82604dced None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.233495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/681] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:39.235534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.235821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.236084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.239021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/682] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:39.240152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.262053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:39.263270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:39.345137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/683] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:39.350484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.360604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.368197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce4e60-acf9-4685-981d-2887e426d1fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.407740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.408400 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.408675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.408943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.416405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.417855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:39.430475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:39.431381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:39.432298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/684] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:39.432783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.460665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.473809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e764-893f-4432-a0ec-3c99c5731f8e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.510801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.511788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.512297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.512785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.541397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d98b-74c8-4a9e-8ed2-130fb4d48225 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.579524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.580149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.580524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.580790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.586943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.588965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.591461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:39.596904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:39.597860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:39.598688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:39.602858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/685] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:39.603698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.623642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.669769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.707369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/686] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:39.708616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.714990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4097-5fd0-4bf1-865f-a8fcbd293636 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.751232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.751838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.752118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.752452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.753144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:39.753442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.754903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.767792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/687] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:39.768717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.817026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e44e-6115-4d2c-9c95-a06df33786e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.849577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.858724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.859698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.860139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.860584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.861431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:39.861951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.863488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.872982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/688] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:39.873969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.878662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ae160-c09d-49c3-bbca-d32f5022fcc9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:39.908030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:39.921034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:39.921828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:39.922649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:39.922928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:39.934288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:39.935772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:39.951085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:39.952230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:39.953329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:39.953329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:39.953329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:39.957684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/689] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:39.958394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:39.989057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00253a8f-e057-4d62-b45d-fc82adc03536 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.025364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.026273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.026762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.027197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.028140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f3216e5d1c354d8e8d5c1fbf449f882c) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:40.028637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.030334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.037434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/690] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/role_assignments?user.id=f3216e5d1c354d8e8d5c1fbf449f882c&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:40.038454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.066281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344ecef-caf5-4f67-a132-e4685e96120a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.095874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.104691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.105268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.105551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.105856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.118440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.119989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:40.135708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:40.136766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:40.137715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:40.137715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:40.137715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:40.142505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/691] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:40.143918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.153320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.160826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d7e9-6068-4a70-9ad4-dddc56b9a458 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.200902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.201856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.202312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.202745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.203705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:40.204172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.205779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.260577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03de94-1a0a-4949-9b12-710ae6f18abe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.301623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.302248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.302519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.302785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.303502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:40.303801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.305341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.326820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/692] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:40.328207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.358300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/693] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:40.374803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.380728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.453303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fb00a-ecd5-4bd6-a0c8-7615411e802b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.495646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.496876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.497147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.497408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.506495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b676df-fdaa-439e-b103-d7cb1e26bc2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.509293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.510998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:40.525203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:40.526152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:40.527052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:40.534818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/694] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:40.536384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.544023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.544686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.544955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.545216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.554119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.556159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:40.571274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:40.572181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:40.573726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/695] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:40.581604 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.610865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.655482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.660484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c7ecc-8cd8-44bf-a5eb-ab7be3645684 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.700454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.701357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.701885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.702290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.703919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:40.704390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.705951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72504a-d97e-4c41-b614-a5b2e4418785 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.707393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fe888-b640-4935-8676-85b3d7110116 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.731522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/696] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:40.733164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c27b-cb3e-450b-b901-d6652381d463 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.733588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c27b-cb3e-450b-b901-d6652381d463 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.734014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c27b-cb3e-450b-b901-d6652381d463 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.736626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/697] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:40.737483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7c27b-cb3e-450b-b901-d6652381d463 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.748123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.748889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.749491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.749491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.750909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:40.750909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.752566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87856909-d807-41be-a1c3-a29a1f340560 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.765552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/698] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:40.767282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.767282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.767747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.786477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:40.787805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:40.850795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7c27b-cb3e-450b-b901-d6652381d463 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:40.864634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/699] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:40.865605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:40.874101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.892473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:40.893444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:40.893892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:40.894345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:40.929375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:40.976680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:40.978578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:40.994280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee640b-38de-4d65-997d-ef950fe2cec9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.028237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.029006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.029348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.029671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.030654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f55820319ba442aa8288cf179681b2d9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:41.030783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.032481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.038843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/700] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/role_assignments?user.id=f55820319ba442aa8288cf179681b2d9&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:41.039806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.096659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/701] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:41.105780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.177220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.202544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.233645 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c845-0e7a-4a51-b766-a09c4f34626c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.261816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096f35-0b28-46e7-bba5-e41075af6b3e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.288946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.289690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.289959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.290219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.290796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.292155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:41.305075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:41.306245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:41.307719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/702] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:41.308797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.310937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.310937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.310937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.312233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.324198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.326652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:41.346665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:41.347639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:41.348548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:41.348548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:41.348548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:41.349211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/703] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:41.355130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.429073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f0ac8-2d01-4eac-9e14-d794061c28ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.468042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.468868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.469216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.469418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.470012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:41.470291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.472131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4b2dd-7f24-4bfa-824f-27396fcc5786 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.475503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.478852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/704] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:41.480739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c5d38-e421-4a38-9993-d3e8dfcdc3eb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.481018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c5d38-e421-4a38-9993-d3e8dfcdc3eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.481289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c5d38-e421-4a38-9993-d3e8dfcdc3eb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.483405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/705] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:41.485251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ce53-a523-4b4e-94cf-50f259268ced None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.486885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c5d38-e421-4a38-9993-d3e8dfcdc3eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.498938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.533639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.535076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.535548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.536520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.537640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:41.538170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.540772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7731b7-3854-437f-a7ab-791320beaec2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.558088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/706] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:41.559881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.560380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.561528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.585634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:41.587556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:41.606370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c5d38-e421-4a38-9993-d3e8dfcdc3eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.658247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.658859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.659132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.659395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.660026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:41.660342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.661708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.672321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/707] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:41.672321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.676365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/708] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:41.698060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.832928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.838083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0eaa3-bb55-4439-b55b-fe567960d0e1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.842658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:41.882331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.882965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.883241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.883537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.899165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.901549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.907623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e698c-d216-4f5a-9cad-4c2d052c06a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:41.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:41.923976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:41.925203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:41.925203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:41.925203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:41.925203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/709] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:41.930485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:41.954767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:41.955316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:41.955589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:41.956104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:41.965082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:41.967012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:41.982506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:41.985335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:41.987463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/710] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:41.994423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:42.061690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee7c9-1c06-4233-bc7f-eb1fe62dc8de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:42.132803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:42.133423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.133699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.133962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.134617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:42.134913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:42.136297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:42.168616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/711] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:42.173993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:42.186343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae05fe-102b-451a-8b7b-7a16820f6a4c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:42.207631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:42.211692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:42.224353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:42.224964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.225229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.225497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.227053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:42.227377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:42.228888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:42.243038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/712] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:42.248791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:42.366446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506acc29-8def-4f38-a8d5-f33e50b4c075 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:42.369825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a2f76-9e16-4ec0-b4a4-b367a05a0ed2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:42.429577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:42.430260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.430453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.430719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.431436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:42.431742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:42.433188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09d4ab-c202-4619-ae88-193bfbe84b94 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:42.435316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:42.435932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.436205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.436492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.437136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:42.438923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa690f7-4eea-4891-81b0-cf2756ecbf99 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:42.451842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/713] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:42 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:42.476200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/714] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:05:42 2022] POST /identity/v3/users => generated 254 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:05:42.708549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929fe2-611e-4dac-8c43-c8a64a916f80 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.708921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929fe2-611e-4dac-8c43-c8a64a916f80 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.709157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929fe2-611e-4dac-8c43-c8a64a916f80 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.710732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/715] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:42.715962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.716439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.716517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.735250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:42.736356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:42.800796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:42.809948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/716] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:42.829998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929fe2-611e-4dac-8c43-c8a64a916f80 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:42.881695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:42.953985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929fe2-611e-4dac-8c43-c8a64a916f80 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:42.991607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:42.992164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:42.992479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:42.992747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:42.993327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:42.994689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:43.003891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:43.004942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:43.005768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/717] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:42 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:43.012846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:43.124934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501abcc-9b91-4d81-95e7-50a19aff70ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:43.162901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:43.163513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:43.163791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:43.164054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:43.164712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:43.165011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:43.166379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37548a1-eac9-4716-bba9-d1ab6cce1737 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:43.171122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/718] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:43.176661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:43.320106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552b8b6-69ab-4c82-bd1f-86f5c94a395d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:43.337754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:43.369090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:43.369713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:43.369988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:43.370256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:43.370905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:43.371205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:43.372605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee41cf4-ed9e-4a69-b15b-7b637cfb64ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:43.380155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/719] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:43.628418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c056a40-331f-4fe2-bad2-af610c27ffea None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:43.628609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c056a40-331f-4fe2-bad2-af610c27ffea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:43.628875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c056a40-331f-4fe2-bad2-af610c27ffea None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:43.630555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/720] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:43.635965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:43.636442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:43.636535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:43.662859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:43.662859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:43.745564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/721] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:43.780603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c056a40-331f-4fe2-bad2-af610c27ffea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:43.855267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:43.897552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c056a40-331f-4fe2-bad2-af610c27ffea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:43.933549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:43.934178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:43.934453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:43.934720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:43.945467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:43.950352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:43.951238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:43.952212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/722] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:43.957519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.068529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21905d-1462-41ab-bf03-eb8588d6a79a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:44.071179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:44.114032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:44.114648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.114926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.115213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.115878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:44.116174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:44.117591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:44.127120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/723] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:44.131736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.243944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8870-91a5-4222-9416-a4ac981bb33d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:44.268451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:44.281161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:44.282147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.282626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.283117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.294145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:44.295945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f0a2c33-84f8-4846-85d2-5de2715c2fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:44.311392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:44.312568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:44.313447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:44.313447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:44.313447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:44.315589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/724] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:44.321085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.376594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40258-ca93-4fdc-9a78-c083c1e0c6fc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.377244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40258-ca93-4fdc-9a78-c083c1e0c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.377771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40258-ca93-4fdc-9a78-c083c1e0c6fc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.379868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/725] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:44.389861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.390737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.391214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.411653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:44.413218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:44.485345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/726] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 11:05:44.504968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148f5c2-d26c-454f-88a2-d794ffc3b8bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:44.520118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.577013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:44.577636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.577949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.578226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.578880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:44.579181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:44.580580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:44.610753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/727] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:44.615916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.632041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09a5a6-d720-4748-a889-3ea6ef406cbe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:44.661902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:44.669969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:44.670894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.671369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.672032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.673041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:44.674618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b1a69-5f00-4c43-8ee1-9db9db1904dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:44.686217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:44.700466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/728] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:05:44 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:05:44.808900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71408880-4e58-404a-adc8-46b14ca848cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:44.874548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:44.875717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.876197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.876687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.888687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:44.889403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796bc3a-79d6-4316-8491-feb84a11114a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.889872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796bc3a-79d6-4316-8491-feb84a11114a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.889951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796bc3a-79d6-4316-8491-feb84a11114a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.891405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:44.891674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/729] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:44.897985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:44.898302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:44.898529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:44.908840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:44.909855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:44.910817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:44.916324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/730] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:44.922874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:44.931105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:44.932740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:45.033915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/731] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:45.038430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.054748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.067570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1d833-b792-4839-8df8-0bc6a1450d2f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.113793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.114472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.114754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.115025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.115768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:45.116075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.119521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:45.156353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/732] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:45.162785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.224969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.236130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88c3cf-6998-45c4-b88d-19e85b89edf6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.294395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7370ff-a922-4aa2-8e36-14d9b2d4ab09 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.307640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.308753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.309237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.309797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.330317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.330850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.331130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.331410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.337211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:45.342106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:45.343015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:45.343900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:45.345478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.348252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/733] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:45.353322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.402759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.404213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:45.505952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/734] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 11:05:45.511282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.543178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a1c03-2eea-43aa-b1ca-95de03b73c34 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.579889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.646718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9c30d-1cea-4323-ad4c-5364a2e3df65 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.686184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.686826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.687430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.687430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.698475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.700088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:45.710328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.711475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.711947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.712433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.713349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:45.713886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.715459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95218f-ed26-40a2-acdb-af04cceb1bec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d939c747-7cc9-42dc-8caa-5ff8fb3063be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:45.715567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:45.716537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:45.718023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:45.718023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:45.718023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:45.718023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/735] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:45.718269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d4c1a-faf4-4d2a-b60c-f6271ad15676 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.718541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d4c1a-faf4-4d2a-b60c-f6271ad15676 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.718816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d4c1a-faf4-4d2a-b60c-f6271ad15676 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.720439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/736] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:45.722188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d4c1a-faf4-4d2a-b60c-f6271ad15676 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.730346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/737] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:45.732536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.733021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.733483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.753658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:45.754822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:45.765788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.819744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:45.823996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/738] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:45.825010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.846523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d4c1a-faf4-4d2a-b60c-f6271ad15676 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.880912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.881567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.881846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.882117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.882812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:45.883109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.884542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d208e31-ee59-4fdc-9df3-6f9b40c842a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:45.896966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/739] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:45.898024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c88b2-641c-463c-882a-16acccf9e04f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.898279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c88b2-641c-463c-882a-16acccf9e04f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.898594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c88b2-641c-463c-882a-16acccf9e04f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.903923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/740] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:45.903923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c88b2-641c-463c-882a-16acccf9e04f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:45.950500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5280437-3e84-4362-b2ab-91014c040721 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:45.984474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:45.985096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:45.985365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:45.985647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:45.995611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:45.997081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.004527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:46.012949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:46.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:46.014794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:46.014794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:46.014794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:46.014794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/741] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:46.014794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.029187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c88b2-641c-463c-882a-16acccf9e04f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.066465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.067171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.067379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.067621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.078976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5decee66-5705-4ce1-a67a-c7fb55bf0ce9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:46.083658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:46.084648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:46.085484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/742] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:46.086197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.086805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.086805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.105322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:46.106511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:46.116311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.132480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760800e7-419f-4e1f-a8bc-cbc0f28c39ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.172093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.172514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.172795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.173070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.192304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/743] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 11:05:46.192304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.194372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.196016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:46.209932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:46.211032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:46.212076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/744] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:46.212841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.279436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.312295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c769c5-eb9e-461a-9f3a-100506e6f1c4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.338106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fb8f8-78ea-4b63-8d24-9c58a483d50a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.350977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.351798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.352070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.352549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.353186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:46.353497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.354853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.369241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/745] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:46.370068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.373025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.384579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.385270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.385511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.385730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.386396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:46.386694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.388166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.403754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/746] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:46.403754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.485456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9448dd-56e1-4d13-9d87-7069071d9d2e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.511713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982a457-32f2-4e1a-b992-c63e88379517 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.522526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.523135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.523407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.523675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.524332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.525701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.534175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.543790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/747] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:05:46 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:46.545662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.571164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.571966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.572337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.572659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.573398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:46.573699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.575506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.594240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/748] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:46.595510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.642184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.693949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dd05c-f1a5-4a32-b477-ed05efce1180 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.737453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.738011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.738285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.738549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.750570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.751209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.760508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774444-328b-42d3-98e1-e0e2c8874a49 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:46.766623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:46.767603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:46.769033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/749] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:46.769728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.807415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.808048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.808412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.808658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.820912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.822405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:46.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:46.846684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:46.848134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:46.848134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:46.848134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:46.851662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/750] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:46.857830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:46.895002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e75c26-119f-40f7-9509-b79782c0cc0f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:46.900911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.937787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:46.940097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:46.940831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:46.941209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:46.941552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:46.942363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f55820319ba442aa8288cf179681b2d9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:46.942736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:46.944295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:46.950801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/751] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/role_assignments?user.id=f55820319ba442aa8288cf179681b2d9&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:46.951739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.010503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af77da8-ae71-4f2e-a72f-23d499eb24bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.049462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.050412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.050855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.051292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.052202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:47.052721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.054335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.073736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/752] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:47.085409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.124291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce1048-cd38-416b-90d4-22f5ca6ad9f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.188955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.189689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.190024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.190356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.191070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:47.192017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.193773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44a692-de40-44eb-bfeb-6eba3cbdabd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.213615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/753] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:47.214892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa542f-3d9a-47d2-bde0-eef713dcc55c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.215172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa542f-3d9a-47d2-bde0-eef713dcc55c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.216058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa542f-3d9a-47d2-bde0-eef713dcc55c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.220655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/754] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:47.224241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.224564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.224829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.238223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.244472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:47.245710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:47.260464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a4964-5050-4873-a5ad-8963032ee4d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.301841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.302800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.303281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.303732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.314311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.318526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.321472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:47.341369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:47.342334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:47.343193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:47.343193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:47.343193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:47.345310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/756] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:47.346394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.348498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/756] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:05:47.351536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.473454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204bce-b75c-4764-8a92-7349ab8dd73c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.496763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7ad4a-6148-4fad-908e-f3c0f1e571df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.509544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.510162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.510725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.510725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.511358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:47.511651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.513052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.530066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/757] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:47.531262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.544911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.546176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.546655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.547351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.559079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.561487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.564404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.571299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:47.581628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:47.582553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:47.583472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:47.591597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/758] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:47.591597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.647062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428dc1-9657-44b1-9e37-f07134b5003d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.686213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578346-df72-44a2-a63d-16a764935009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.687256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578346-df72-44a2-a63d-16a764935009 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.687724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578346-df72-44a2-a63d-16a764935009 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.688215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578346-df72-44a2-a63d-16a764935009 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.689013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578346-df72-44a2-a63d-16a764935009 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.690579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98578346-df72-44a2-a63d-16a764935009 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98578346-df72-44a2-a63d-16a764935009 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:47.699475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:47.700893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:47.701703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:47.708309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dafd7-cfb7-4a68-a692-11c411763dc9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.708876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/759] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:47.709913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578346-df72-44a2-a63d-16a764935009 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.739462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.740126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.740429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.740688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.748751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.750448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:47.761546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:47.762452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/760] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:47.763724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.824919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.828888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578346-df72-44a2-a63d-16a764935009 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.829509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:47.867472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.868594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.869091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.869551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.870610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:47.871193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.872863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.873824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f912bb-8e67-4448-8d21-5b531f6906a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:47.891964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/761] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:47.893199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:47.916115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:47.917021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:47.917337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:47.917696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:47.918415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:47.918805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:47.920277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:47.925786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/762] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:47.926690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.007148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcc13-ff51-4ca2-89f0-2d7d68727393 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.034528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61692bd0-f354-44b2-9e27-e35b2c320c4f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.043886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.044767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.045187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.045674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.046518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:48.047013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.048713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.058340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.061974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/763] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:48.063323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.080962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.081894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.082285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.083106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.086606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.176598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.178231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.178349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca13b3-845b-4b22-9922-0d7ea368d1f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.225537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.226559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.226840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.227114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.228037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:48.228359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.230058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.236923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/764] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:48.238264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.290621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/765] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:05:48.291724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.317900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.347540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.372933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228d27-ef42-406e-a19e-49c415a2eb00 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.403894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be6798-2131-4f7a-86ea-9c8307c49990 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.416365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.417052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.417330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.418011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.420534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=fbbd789ff1f244eb98d2e5dd001a81c1, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.422811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.440952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/766] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:48 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:48.441778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.453466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.454165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.454438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.454826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.455582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] RBAC: Adding query filter params (interface=public, service_id=e15403cf3fef48b8ab27b59ccb5e211c, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:48.455877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.457355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.474266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/767] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/endpoints?service_id=e15403cf3fef48b8ab27b59ccb5e211c&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:48.479478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.560449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffc962-0439-4a7a-b00d-b9c037446a3b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.602968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.603566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.603846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.604119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.617897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.619663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.621843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:48.639968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:48.642593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:48.644732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:48.644732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:48.644732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:48.644732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/768] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:48.649515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.675080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.695415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a237e4-312c-44a8-9ecf-cb703dd30cbd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.738381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.739393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.739943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.740224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.740889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.742847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b267c-b919-44ef-a0eb-31f65cb6fe3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.750025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/769] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:48.751688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da86e2-9221-4a13-a459-4e343382d2da None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.751969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da86e2-9221-4a13-a459-4e343382d2da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.752239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da86e2-9221-4a13-a459-4e343382d2da None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.754100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/770] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:48.766044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.766365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.766579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.813450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:48.814713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:48.816706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77866bc5-7286-48c4-b1b6-fd8ad8a0e81f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:48.865207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:48.866678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:48.866951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:48.867217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:48.867880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:48.868181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:48.869614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:48.895189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/771] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:48.901152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:48.957881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/772] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:48.992555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:48.995519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.061135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b45109-98df-4abe-8bc8-5f25eb724b82 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.095353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.101368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.102247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.103477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.103908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.112112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.118031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.118188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8fc7a-295f-4485-af12-eb116447c419 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:49.134619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:49.135730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:49.136844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:49.144902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/773] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:49.152023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.156726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.157460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.157745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.157983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.158561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.159917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:49.170302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:49.171547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:49.172703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/774] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:48 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:49.179519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.290146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb20bc-80ab-4533-b1a9-1c57b254ad9c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.292923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa76af-6cd6-4736-a05e-fb4d92ebf9e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.305306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.327264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.328136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.328316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4691f40-f058-4367-b499-18962245d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.328555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.328726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.328968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691f40-f058-4367-b499-18962245d977 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.329245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691f40-f058-4367-b499-18962245d977 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.329402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.329560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4691f40-f058-4367-b499-18962245d977 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.329704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.330173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691f40-f058-4367-b499-18962245d977 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.330518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691f40-f058-4367-b499-18962245d977 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.330927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1244b7-6956-48fc-87a7-341b1138c445 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.332066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4691f40-f058-4367-b499-18962245d977 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.336285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/775] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:49.337464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2905-3798-4004-bd53-734ea7612543 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.337737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2905-3798-4004-bd53-734ea7612543 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.338000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2905-3798-4004-bd53-734ea7612543 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.339390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/776] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:49.345264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e2905-3798-4004-bd53-734ea7612543 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.349920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/777] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:49.351227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.351502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.351768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.363053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.372478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:49.373589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:49.453542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/778] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:49.454636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.485559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e2905-3798-4004-bd53-734ea7612543 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.530945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.531566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.531820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.532078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.533153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.533446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.535326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.542891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/779] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:49.543985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.579719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdf91b-061e-4858-b53e-a8b09a2aeef8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.623206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.624167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.624715 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.625185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.626136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=baa77a1e4ec94a159eccbee7db74bcf4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.626641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.628385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.634576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.635482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/780] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/role_assignments?user.id=baa77a1e4ec94a159eccbee7db74bcf4&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:49.636602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.675980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c754fb-b091-4c18-91e7-f7a9e9c6e22b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.717180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.717995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.718266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.718579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.731561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:49.737470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:49.738489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:49.739332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/781] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:49.744946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.754820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efd06a-5651-49a9-bb8a-286e5ffb4ae3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.791947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.792957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.793436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.794045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.795009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] RBAC: Adding query filter params (interface=public, service_id=bb20e21b50e14c86b6c470b7895d93e1, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.795525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.797353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.806171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/782] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/endpoints?service_id=bb20e21b50e14c86b6c470b7895d93e1&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:49.813429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.850238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.857575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302c97e-0597-4f25-b3ea-9a94513f39dc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.916412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:49.932383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840329c-ac86-4a49-ad82-b510a798bb7a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:49.933155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.933778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.934048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.934312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.934983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:49.935276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.936784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.957506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/783] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:49.963060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:49.970318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:49.971060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:49.971404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:49.971793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:49.972496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:49.974033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:49.984314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/784] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:50.089695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2129e-0cb7-4466-82cd-e10f9f64045b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:50.132378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:50.133028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.133285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.133609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.149514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:50.151009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:50.173498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:50.174464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:50.175987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:50.175987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:50.175987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:50.175987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/785] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:50.180755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:50.281827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:50.288863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055e914-668f-4e77-b68a-2a6c3e7e0fb9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:50.327107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:50.327773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.328188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.328484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.329129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:50.329422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:50.330769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:50.345585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/786] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:50.352522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:50.421727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:50.476876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab90905-b1d9-4d3a-88c9-b069b5d5eb9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:50.510446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:50.511065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.511334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.511595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.524830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:50.526269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b4d8d4c2-8d55-46fe-95ca-009edb5a7635 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:50.539566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:50.541850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:50.544039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/787] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:50.548771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:50.686122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c0a91-5e5f-4df7-b359-fa9079c27c08 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:50.736160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:50.736716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.736987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.737174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9342d-d45d-487c-bbe1-a92198ae8376 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.737353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.737500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9342d-d45d-487c-bbe1-a92198ae8376 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.737642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9342d-d45d-487c-bbe1-a92198ae8376 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.737966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:50.738260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:50.739328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/788] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:50.739680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:50.744259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:50.744566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:50.744838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:50.763504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:50.764703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:50.766989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/789] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:50.773605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:50.790180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:50.888004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/790] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:05:50.906742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c736269-096d-44f3-84aa-321c799cecb8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:50.908924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:50.967142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.013884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.014517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.014786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.015048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.021113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb1245-7a06-45e3-8527-113835e37488 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.045339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.046990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.062129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.062766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.063217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.063592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:51.076777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:51.077835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:51.079251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:51.079251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:51.079251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:51.079251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/791] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:51.088821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:51.093494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:51.095615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:51.097783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/792] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:51.099379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.104393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.184976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.224167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78ccb-30ae-4864-b6ea-9aa07c52d334 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.233651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.250453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f69262-6131-4f9f-bdfa-e265299f32ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.267192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.267785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.268064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.268355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.269005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:51.269306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.270988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.284046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/793] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:51.289501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.349109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.350034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.350509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.350944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.351777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:51.352243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.353798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.368863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/794] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:51.377990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.405361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191ef17-ea7a-416e-9cf8-b9b1afcd7844 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.432775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.442996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.443620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.443898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.444207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.451192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.452719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:51.464164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:51.465086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:51.465948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/795] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:51.471477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.548638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c48228-74ba-409e-ab3f-55a1f77c4c30 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.595637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.596708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.596987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.597256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.602931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f01c-d918-475c-9d0d-faebe54bcdc4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.611120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.612663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6e9c4569-d460-4a87-af16-bf033470c511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:51.632215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:51.633218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:51.634640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:51.634640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:51.634640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:51.636396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/796] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:51.637884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6ec8-9847-440b-84b6-1198b714a577 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.638350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6ec8-9847-440b-84b6-1198b714a577 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.639604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc6ec8-9847-440b-84b6-1198b714a577 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.641659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/797] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:51.642371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc6ec8-9847-440b-84b6-1198b714a577 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.649488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.650390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.650892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.651372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.652746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:51.653255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.655259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae34ebf-ac2a-409f-b4ce-eead8dddbf9d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.655790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.676490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/798] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:51.676490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.676490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.676490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.696811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:51.697990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:51.742681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.769575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/799] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:51.770593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.786340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc6ec8-9847-440b-84b6-1198b714a577 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.824962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.825876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.825876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.826179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.826926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:51.827299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.828986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.841579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/800] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:51.847464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.884952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefec92-d911-44a7-be2f-5ffb3f3b0106 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:51.920429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:51.921092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:51.921636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:51.921968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f55820319ba442aa8288cf179681b2d9/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:51.924406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=f55820319ba442aa8288cf179681b2d9, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:51.925646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:51.927515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:51.946297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/801] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:51 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/f55820319ba442aa8288cf179681b2d9/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:51.946297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:51.998724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7875d26-0b64-4c06-8012-5ebe7d5b9ac9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.040828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.082586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.083145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.084478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.084762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.085354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.086748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-35b7dc4b-9e9d-44ee-a802-ff7ddabd9104 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:52.113468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:52.114540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:52.115393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/802] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:52.121212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a90b6-8d9c-459a-aa6b-f9b6054b907a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.121366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a90b6-8d9c-459a-aa6b-f9b6054b907a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.121571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a90b6-8d9c-459a-aa6b-f9b6054b907a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.123886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/803] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:52.124740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a90b6-8d9c-459a-aa6b-f9b6054b907a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.140622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cef0a-9b91-4279-83ed-23834e9c133b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.181734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.182356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.182625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.182891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.190894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.192252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-07f46ea7-3b01-4942-b985-e0831e664475 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:52.205197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:52.206173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:52.207123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/804] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:52.207704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.208291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.208291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.226439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:52.227608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:52.259062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a90b6-8d9c-459a-aa6b-f9b6054b907a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.296644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/805] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:52.297114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.297609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.298282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.298753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.299273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.300115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:52.300644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.302214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.303585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.308070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/806] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:52.314493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.347320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.417422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e545fed-8bf0-4ff5-a24f-9c5256181b7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.421133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71397b-6e1e-434d-8f0f-d7ed06be79b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.453974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.455033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.455508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.455999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.456884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.457022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:52.457529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.457649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.457893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.458226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.459010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:52.459367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.459968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.460905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.470329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/807] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:52.471491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.479328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/808] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:52.480353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.537559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.566302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.582185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b486fe8-ee3f-4e57-b9fd-13aae569ed7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.583087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465229b-2c95-482c-96da-e880d57e21a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.619591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.620087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.620209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.620403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.620672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.621324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.621585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.622020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.622451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.622690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefa5a-afe1-4576-bd89-b98d3b361431 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.634290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:52.639953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:52.647244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/809] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:52.648216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.656532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/810] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:05:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:52.656532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace750c3-78b4-4fe0-bae2-28dc666e6c16 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.656532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace750c3-78b4-4fe0-bae2-28dc666e6c16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.656532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace750c3-78b4-4fe0-bae2-28dc666e6c16 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.656532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/811] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:52.657868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace750c3-78b4-4fe0-bae2-28dc666e6c16 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:52.781545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108bea9-4664-4c46-b9ff-b3d445d7c603 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.813581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.814784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace750c3-78b4-4fe0-bae2-28dc666e6c16 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:52.829438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.830059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.830336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.830604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.857155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:52.858022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.858296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.858711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.860234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:52.860736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.862100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c00b6b-e453-4a0c-af8d-2b5014b1d8c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.864981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:52.877257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/812] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:52.878449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:52.878730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:52.878999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:52.899180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:52.900421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:52.927333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:52.929664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:52.988733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/813] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:52.990303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.067800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/814] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:05:53.068707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.094995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.125785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539a3d0-c27e-4413-93d5-eb0e65e36c8c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.160362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.169304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.169935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.170209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.170481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.183702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.184487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432889c7-976e-4470-bdaf-55b514d33f80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.185190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:53.203070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:53.204157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:53.205175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:53.205175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:53.205175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:53.206925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/815] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:53.208020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.231069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.231695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.231967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.232232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.232859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.234608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:53.244250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:53.245488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:53.246715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/816] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:53.247502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.323482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288ff86-dd0d-4d2d-8430-06945b97f20f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.340060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.357712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.358479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.358838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.359335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4dfa-1e5c-42de-a6d0-862f08b86bd8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.359790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.370444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.372056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:53.386487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:53.387635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:53.388732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:53.388732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:53.388732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:53.393916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/817] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:53.394856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.400437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.401222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.401638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.402065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.402907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:53.403340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.404867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.423897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/818] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:53.425067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.436632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.498335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436353e-9223-409b-b3ca-1aa608dc161e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.533327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d6e3-bfdb-4db8-8252-a22feda88695 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.533527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.534079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.534420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.534771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.535472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:53.535864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.537668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.547459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/819] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:53.548356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.571519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.572443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.572891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.573356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.574161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:53.574631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.576236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.588649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/820] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:53.589732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.599577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.656825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.662815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a98adf-cf7b-4d8d-836f-d062ab7ae280 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.699672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.700455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.700832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.701199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.703763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d763632-341d-4b5c-88a4-389f445e4ce8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.710067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.711897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:53.724223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:53.725286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:53.726219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:53.730706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/821] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:53.731656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.744747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.745662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.746208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.746638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.747485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:53.747952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.749813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.756113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/822] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:53.757201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.834695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.850250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5678383-8045-49dc-8ac9-c1688b97d94c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.869842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1df6a-3b8c-4926-a744-083331e29d6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:53.888372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.889096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.889439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.889774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.898145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.899875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:53.914825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:53.915695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:53.916610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:53.918144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:53.919047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:53.919522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:53.919985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:53.920845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/823] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:53.921166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:53.922268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:53.922710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:53.924449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:53.942277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:53.950668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/824] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:53.951916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.028912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147c7a2-b60d-4d54-bb4d-087a6a70e333 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.058517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad423-8854-4f94-810f-79b7c8546545 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.064860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.065490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.065776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.066048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.066741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.068113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.090458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/825] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:05:53 2022] POST /identity/v3/endpoints => generated 337 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:05:54.091260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.092880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.110056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.111168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.111641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.112187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.113166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:54.113694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.115453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff73c0-09a0-4b27-88a0-49e384c18b01 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.130075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/826] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:54.131709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102867-179e-48eb-a23e-4bd5fbd071d2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.132196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102867-179e-48eb-a23e-4bd5fbd071d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.132709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102867-179e-48eb-a23e-4bd5fbd071d2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.134560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/827] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:54.135952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102867-179e-48eb-a23e-4bd5fbd071d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.197878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.236715 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db805f5f-c5d6-46ef-b401-e807dbe333cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.256841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102867-179e-48eb-a23e-4bd5fbd071d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.293513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.294222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.294497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.294769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.300693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.301663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.302162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.302616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.304062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:54.304407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.306223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b3183c-c823-4ae1-86e7-a69bcb7bdacc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.308014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.309502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.314889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/828] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:54.316124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.316455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.316720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:54.333670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:54.334686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:54.335550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:54.335550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:54.335550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:54.335550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/829] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:54.341207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.343830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:54.345907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:54.403407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.435544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/830] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:54.460538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.484824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aeacfa-5d51-4fb6-8efc-a8a889f73582 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.516527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.530841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.531713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.531993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.532281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.532933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:54.533229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.535067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.549974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/831] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:54.555714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.575235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47484356-3317-4785-9fc0-b4d8a526ad1b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.620585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.620585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.620968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.620968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.634127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:54.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:54.640157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:05:54.641850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:54.644310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/832] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:54.650196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.673585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375126-2767-48a5-91ba-183d420dd040 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.696405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.716907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.717527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.717796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.718057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.726288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.727574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:54.739852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:54.740817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:54.742091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/833] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:54.747149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.778150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d42f5-3bc2-4789-8b7c-973bd9cfe398 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.809650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.819305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.819915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.820189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.820479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.821134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:54.821429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.822791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a86b6a-4c7f-4244-8025-f4ccda9d8a39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.832215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/834] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:54.833723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a091f-6cdd-48c5-b406-b66f53f2545f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.834127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a091f-6cdd-48c5-b406-b66f53f2545f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.834454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a091f-6cdd-48c5-b406-b66f53f2545f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.835797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/835] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:54.840710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a091f-6cdd-48c5-b406-b66f53f2545f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:54.863834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e34d7-22a8-4a4c-83af-cb092d3afee8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.898430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.899038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.899316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.899588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.900294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:54.900595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:54.902009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f157ffe-547e-4396-9762-2fcd152b8be6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:54.920312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/836] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:54.920312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.920312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.920312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:54.938058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:54.939239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:54.954908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:54.959579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a091f-6cdd-48c5-b406-b66f53f2545f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:54.996145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:54.997064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:54.997334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:54.997600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.008220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.020494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/837] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:55.020965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.021070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:55.037321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:55.038250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:55.039141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:55.039141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:55.039141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:55.039141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/838] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:55.040036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.057792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:55.125961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede712-a63f-40d3-a3d5-8e2fc2867f51 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.152098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af574c4c-6e66-44a2-bb39-efd23e3562bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.163927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.164620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.164893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.165151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/baa77a1e4ec94a159eccbee7db74bcf4/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.180581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=baa77a1e4ec94a159eccbee7db74bcf4, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.182021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.186860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.187831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.187948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.188160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.188870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.190219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:55.198982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:55.199961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:55.200837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/840] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:55.201628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.204363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/840] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:05:55 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/baa77a1e4ec94a159eccbee7db74bcf4/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:05:55.209089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.238619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:55.315144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:55.337528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f01c-e0b3-421e-be18-b0ad5844b09c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.364101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65323b-bd5f-402a-8519-bcf1db1441d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.385781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99195628-6490-4984-835b-901cef32f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.386392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195628-6490-4984-835b-901cef32f837 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.386670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195628-6490-4984-835b-901cef32f837 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.386938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195628-6490-4984-835b-901cef32f837 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.387924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99195628-6490-4984-835b-901cef32f837 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:55.387924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99195628-6490-4984-835b-901cef32f837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.389312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99195628-6490-4984-835b-901cef32f837 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.396584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/841] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:55.402098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99195628-6490-4984-835b-901cef32f837 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.412535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.413456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.414276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.414745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.415936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:55.416726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.418452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.435502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/842] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:55.441192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.504024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99195628-6490-4984-835b-901cef32f837 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.544444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:55.546976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.547662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.548030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.548399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.549162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:55.549494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.550927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.564911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/843] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:55.578369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ce251-d25b-445b-b0c1-6cc3b4f152da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.616908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.617638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.617905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.618173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.627200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.628712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:55.642670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:55.643667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:55.645293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/844] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:55.653406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.672260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:55.771076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedac5-a54c-4f62-aa3d-544f6dcd8063 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.806599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.807202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.807475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.807742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.808495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:55.808799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:55.810213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:55.829223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/845] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:55.834906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:55.942482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e673c-c17c-4220-9671-b699d13d5e1b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:55.976770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:55.977373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:55.977645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:55.977906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:55.981484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.047316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.048813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.157062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/846] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:05:56.162987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.232619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.263005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18cc68-3d2a-49b7-930c-3bd27ad41947 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:56.297897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:56.298584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.298790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.299045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.309094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.310512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:56.325023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:56.326029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:56.327590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:56.327590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:56.327590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:56.327590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/847] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:56.332316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.368159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.436349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469bf3b-9d9a-41ae-adf6-c5cad786ce33 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:56.470559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:56.471160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.471429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.471688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.472339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:56.472632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.474055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c826b-e7dc-430b-9fcc-cd7d8fb45da5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.485625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/848] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:56.493376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.574565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd165-ac67-4f69-90b1-66a34a90e653 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.574846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd165-ac67-4f69-90b1-66a34a90e653 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.575116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cd165-ac67-4f69-90b1-66a34a90e653 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.576696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/849] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:05:56.582537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.583363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.583803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.595485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb14b2-288d-42a8-8eb4-4023f00b6769 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:56.606691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:56.608335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:56.622246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.634827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:56.635505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.635793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.636037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.646961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:56.660820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:56.661872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:56.662767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/850] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:56.667494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.708391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/851] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:05:56.729334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.766675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.778631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a0d9-af43-42b4-9c09-e87108bdb10e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:56.814674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:56.815882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.816435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.817251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.818366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:56.818861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.820879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.838986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/852] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:56.844931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.855883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8abd0-bcba-47d7-8bef-9aa10d743a44 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:56.895067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:56.896356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:56.897343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:56.897809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:56.898556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:56.900931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:56.901975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:56.913904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:56.914865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:05:56.915725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:56.924193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/853] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:56.925358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:56.971766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3749d5b-2be1-4668-94ed-3589632e9c3e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.006973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.007597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.007875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.008185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.009008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=f55820319ba442aa8288cf179681b2d9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:57.009231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:57.010651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:57.017046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/854] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/role_assignments?user.id=f55820319ba442aa8288cf179681b2d9&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:57.034902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:57.047248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011c123-b3c1-4b8a-8ed5-e23738d7bb09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.091685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.092721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.093285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.093747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.094603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:57.095098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:57.096734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47258bdf-26c2-4ab4-b968-acc01b551e75 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:57.102520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/855] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:05:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:57.108622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:57.217522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01471b-d5ec-4601-9dd1-aaea026d6d0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.225570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:57.259259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.259880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.260154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.260450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.261094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:57.261396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:57.262758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f2ee-2d7c-4d1f-a33e-6dad9ee925c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:57.268152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/856] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:05:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:57.306498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6683f0-72f2-44ca-8361-bb1c9add107a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.306772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6683f0-72f2-44ca-8361-bb1c9add107a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6683f0-72f2-44ca-8361-bb1c9add107a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.308641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/857] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:57.314276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.314570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.314825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.334649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:57.335877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:57.410065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/858] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:57.433505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6683f0-72f2-44ca-8361-bb1c9add107a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:57.541714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6683f0-72f2-44ca-8361-bb1c9add107a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.582966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.583656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.583934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.584238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.595981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-82ef739d-2f20-4ac4-aa18-459fd45ccfeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:05:57.600971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:57.601846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:57.602666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/859] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:57.609977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:57.709712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f710-4a4c-41b7-90f1-99d963407c42 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.710069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f710-4a4c-41b7-90f1-99d963407c42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.710260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831f710-4a4c-41b7-90f1-99d963407c42 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.712791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/860] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:57.716810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.717085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.717357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.718504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a006e4b-7d5a-49b5-9554-74e7b2ac096b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.722279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:57.735352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:57.736578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:57.746361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:57.759452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.760065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.760365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.760636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.761306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:57.761603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:57.762962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:57.780107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/861] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:57.781222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:57.828443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/862] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:05:57.849487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:57.891763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76467b1-c612-4e23-8ae4-0f6508ceefeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.941615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:57.943121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:57.943603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:57.944354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:57.955832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b7d38-6d0f-4d0c-8e76-337f69b3c00c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:57.965660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:57.967912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:57.989228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:57.990214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:57.991207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:57.991207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:57.991207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:57.995167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/863] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:57.997086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:58.002134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:58.003016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:58.003481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:58.003940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:58.004830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:58.006388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:58.015888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:58.028320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/864] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:05:57 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:05:58.117438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525ffae-60d3-459f-a42e-c9f66909fae7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:58.162968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:58.163904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:58.164402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:58.164811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:58.165651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:58.166089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:58.167641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:58.184808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/865] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:58.188849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:58.192770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:58.301780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ca39-3b1f-419f-a33a-07e93e800f11 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:58.337199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:58.338191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:58.338662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:58.339124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:58.348112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:58.349904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:05:58.361149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:58.362056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:05:58.362980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:58.369937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/866] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:05:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:58.371458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:58.442963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:58.475490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b1c5-8dc5-4c5e-a804-821d58e99b11 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:58.509526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:58.510130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:58.510399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:58.510661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:58.511391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:58.511689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:58.513126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:58.535407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/867] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:05:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:58.536864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:58.604903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:58.644076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd956c7a-8171-4eb4-b9ea-1a668a1d5e67 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:58.677894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:58.678496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:58.678766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:58.679088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:58.750700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:58.752562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:58.815331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:58.855557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/868] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:05:58 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:05:58.861057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:58.967203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca98b-7ab0-47b5-94a6-7fb933e3c6c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:58.994188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:59.002178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.003222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.003759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.004465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.014783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.016261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1a82fec3-8d34-4321-96e0-4964eb256640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:05:59.031211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:59.032320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:05:59.033227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:05:59.033227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:05:59.033227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:59.033227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/869] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:59.038158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.126168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3d6d-4118-4fd3-97f2-cd128b35582b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.126491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3d6d-4118-4fd3-97f2-cd128b35582b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.126762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f3d6d-4118-4fd3-97f2-cd128b35582b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.128346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/870] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:59.133454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.133600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.133812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.150034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8871-6351-41bc-b3d8-ce32029e28b3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.152234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:59.153438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:59.187354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.188454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.188867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.189361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.190130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:59.190561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.192934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.206165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/871] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:59.212314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.222020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/872] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:05:59.243393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:59.243589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.336003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598af6e-0272-4516-af28-44e5975afbc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.346677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51fd74-fe4b-4939-b06d-f1de42bfef0e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.369349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:59.379265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.380329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.380710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.381071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.383911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.388002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.388984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.389493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.389600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.390150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.390980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.392920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:05:59.402000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:59.402875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:05:59.403753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:05:59.405186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:05:59.406638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:05:59.407457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:05:59.408171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/873] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:05:59.409634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/874] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:05:59.410517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.415059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.517783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:59.537341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974e36e-dda4-4666-9ea8-7c8fd56750d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.537955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190edc86-50d6-49d9-b6ac-4d2e6cd842ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.573823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.574436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.574772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.575046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.575679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:59.576025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.576680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.577480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.577762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.578227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.578783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.579758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:59.580251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.581934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0a3a-4c98-4ccb-82d1-a5f6bfc44b37 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.587197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/875] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:59.588206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.606242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/876] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:59.606242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85d57-086c-4260-b6f6-671851c0394c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.606242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85d57-086c-4260-b6f6-671851c0394c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.606242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85d57-086c-4260-b6f6-671851c0394c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.606242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/877] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:05:59.609460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85d57-086c-4260-b6f6-671851c0394c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.633727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:05:59.697081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907d360-09c3-4e31-a2b1-57e33c7e0588 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.720050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85d57-086c-4260-b6f6-671851c0394c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:05:59.731488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.732186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.732494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.732760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.733397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:59.734208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.735083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3edf0-ec4c-4104-9294-a7ba7cf35b80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.748297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/878] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:05:59.748297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683385-08c4-4ff6-93ed-c5f1600f6f9d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.748297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683385-08c4-4ff6-93ed-c5f1600f6f9d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.748297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683385-08c4-4ff6-93ed-c5f1600f6f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.765343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ea683385-08c4-4ff6-93ed-c5f1600f6f9d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:05:59.766657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea683385-08c4-4ff6-93ed-c5f1600f6f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:05:59.774173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:05:59.774713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:05:59.774989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:05:59.775257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:05:59.776879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=baa77a1e4ec94a159eccbee7db74bcf4) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:05:59.777282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:05:59.778842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:05:59.786010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/879] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:05:59 2022] GET /identity/v3/role_assignments?user.id=baa77a1e4ec94a159eccbee7db74bcf4&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:05:59.855060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/880] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:05:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:05:59.879266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:05:59.893076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:00.003893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc4b4f-7a6b-4f9d-a83d-3a28189c7306 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:00.040691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:00.041238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.041513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.041780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.042445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:00.044012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780c571-8125-437c-8889-2ae88d39cf5f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:00.067200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/881] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:05:59 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:00.172154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f3ca-07a0-44b2-8acd-cf3ae6cba9f6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.172623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f3ca-07a0-44b2-8acd-cf3ae6cba9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.172985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f3ca-07a0-44b2-8acd-cf3ae6cba9f6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.174680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/882] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:00.180356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.180738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.181069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.201185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:00.202416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:00.272811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:00.287643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/883] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:00.310356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197f3ca-07a0-44b2-8acd-cf3ae6cba9f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:00.341785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:00.415624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197f3ca-07a0-44b2-8acd-cf3ae6cba9f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:00.449845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:00.450468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.450782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.451045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.451619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:00.453507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:00.462315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:00.463288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:00.464152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/884] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:00.469705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:00.570494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c02ff3-862f-410b-8439-348a71ae3cc0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:00.605816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:00.606428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.606702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.606967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.607592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:00.607888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:00.609801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:00.615410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/885] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:00.620994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:00.716162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:00.722132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949695c6-1ec1-4db1-b2bb-efe6d0b4740a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:00.756047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:00.756678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.756953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.757223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.757852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:00.758152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:00.759509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:00.764598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/886] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:00.770238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:00.828667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:00.872105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f4fd8-82b8-4453-ba68-b9f5b5a863eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:00.907344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:00.907993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:00.908290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:00.908566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:00.909271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] RBAC: Adding query filter params (interface=public, service_id=299697c04fc942e2823015e7e1b06d7a, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:00.909575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:00.910955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786683-7071-426e-b491-4cf9cfbca766 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:00.916910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/887] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:06:00 2022] GET /identity/v3/endpoints?service_id=299697c04fc942e2823015e7e1b06d7a&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:00.923192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:01.033679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352dc3-ea60-4fc9-9ffc-407ba95c060f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:01.067805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:01.068494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:01.068773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:01.069043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:01.069649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:01.071001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e31ffb-6992-413a-bbf5-ea591112ff56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:01.077057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/888] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:00 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:06:01.866377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273c3e-1ce9-436f-9bc1-ab0292f40825 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:01.866915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273c3e-1ce9-436f-9bc1-ab0292f40825 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:01.866915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273c3e-1ce9-436f-9bc1-ab0292f40825 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:01.868490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/889] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:01.873964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e011-3c33-4a87-a81e-121c68ee4a01 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:01.874425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e011-3c33-4a87-a81e-121c68ee4a01 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:01.874710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0e011-3c33-4a87-a81e-121c68ee4a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:01.893069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44b0e011-3c33-4a87-a81e-121c68ee4a01 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:01.894251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44b0e011-3c33-4a87-a81e-121c68ee4a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:01.912636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:01.963590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/890] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:01.986281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc273c3e-1ce9-436f-9bc1-ab0292f40825 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.025520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374518e-682a-4c68-9e25-7ec0a61721c1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.026140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374518e-682a-4c68-9e25-7ec0a61721c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.026639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374518e-682a-4c68-9e25-7ec0a61721c1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.028436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/891] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:02.034232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.034760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.035222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.053913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:02.055420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:02.098177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc273c3e-1ce9-436f-9bc1-ab0292f40825 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.129749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.135767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/892] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:02.154963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.155628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.155906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.156167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.156864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.158305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f13aa7-c1be-4430-a7fc-f46b7b102d54 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.163161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.178202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/893] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:06:01 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:02.186806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02663f-e78f-469a-aecc-a92fc9e2c415 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.187066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02663f-e78f-469a-aecc-a92fc9e2c415 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.187367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02663f-e78f-469a-aecc-a92fc9e2c415 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.189079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/894] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:02.194584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.194890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.195112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.215082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:02.216355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:02.283192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.302289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fc96f-8050-4ad8-b24e-9a0538a2386b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.311235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/895] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:02.334028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.346188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.347279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.347557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.348107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.361320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:02.365959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:02.366893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:02.368372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/896] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:02.373805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.461484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd1b5f-605a-452e-ae36-2b4849d99e17 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.486798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb8b55-83b3-41aa-a01a-12f77d67cf13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.498293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.510854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.511489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.511761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.512028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.513172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.515019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:02.526976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:02.527919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:02.529477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/897] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:02.532127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.532836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.533481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.533814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.534445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.534553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:02.534902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.536429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.545808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/898] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:02.551080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.560850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.655855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cf22-fd84-4d22-abb5-62655978bdc6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.687514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb4675-51b8-4f23-8c97-80af11fef189 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.695246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.695856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.696124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.696434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.697531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:02.697531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.698837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.708628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/899] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:02.710645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.726902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.728128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.728793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.729242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.742368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.744228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.748848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:02.762694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:02.763708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:02.764742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:02.764742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:02.764742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:02.766809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/900] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:02.775439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.818689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9368c9-0199-4220-8118-e730792bfd2b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.833379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.856332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.857281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.857800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.858219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.859028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:02.859461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.861246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.871632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/901] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:02.873271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.876800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418742f-054c-4cb2-b9e8-4a06996425c2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:02.911557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:02.912168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:02.912473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:02.912745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:02.913391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:02.913687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:02.915080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:02.932697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/902] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:02.933740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:02.987480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:02.992438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e60fc-bfa9-4b86-a2ef-c0e9d3e2d3bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.029441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.030287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.030946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.031401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.032388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] RBAC: Adding query filter params (interface=public, service_id=52291a372cd9423c8335581dd5ae72d0, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:03.032841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.034363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.034831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8c0aa-fc03-40b0-81e3-0395d4a9bdc4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.044977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/903] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/endpoints?service_id=52291a372cd9423c8335581dd5ae72d0&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:03.046195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.053448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.071239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.071729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.072005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.072293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.080101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.081524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:03.093477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:03.094747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:03.095726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:03.098995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/904] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:06:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:03.099837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.171788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052701-3cb3-4178-9780-ae0f44f84473 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.201463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b03e2a-fa90-414a-b24e-9d1ff7cbc6e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.208785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.209590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.210000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.210489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.211257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.212894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77f513-0c73-4fc6-b07a-bb5264b6dcaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.219375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/905] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:06:03.220843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee4977-5021-4061-8ccd-dc00528ed70c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.221293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee4977-5021-4061-8ccd-dc00528ed70c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.221767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee4977-5021-4061-8ccd-dc00528ed70c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.223184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/906] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:03.228801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.229102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.229343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.243397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.244446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.244926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.245396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.246359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:03.246895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.248611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.249261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:03.250695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:03.272608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/907] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:03.276057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.277232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.316728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.336652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/908] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:03.352862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.389312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0376e-bff7-41ee-969e-81c01bdbc985 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.425486 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.426183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.426463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.426735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.480996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685243d-39bb-499c-9a36-221ddc0933bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.508880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.510674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.527124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.527760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.528029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.528311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.529155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.531034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:03.542893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:03.543857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 11:06:03.544689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:03.548330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/909] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:03.550994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.555204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.585016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.626289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/910] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 11:06:03.631567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.687196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be3a5-7301-4470-8af4-cc13ccbd9be8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.734558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.734558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.734558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.734558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.734981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:03.734981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.736230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.740078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d769e-9236-47cc-ba9f-593e580470e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.756656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/911] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:03.758275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.781452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.782019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.782308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.782820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.786609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.797193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.799121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:03.817641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:03.818709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:03.819588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:03.819588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:03.819588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:03.823559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/912] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:03.824446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.873466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:03.886721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a0aac-6bcd-4c6d-8aca-ecf3c0f6ae63 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.922208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.922693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.922968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.923235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.923882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:03.924196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.925602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:03.935699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/913] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:03.936572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:03.954345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6a5dc-e22f-415c-975a-973a0ef77906 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:03.992036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:03.993162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:03.993692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:03.994249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:03.995095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:03.995577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:03.997387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.011023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/914] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:04.016391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.042402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4926afa-1956-4d93-900d-3cf402960330 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.069981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.077690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.078332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.078613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.078877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.079536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.080951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.112780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/915] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:04.135796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600336-2744-4a94-b9fc-bd3e536d7e64 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.182681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.183784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.184293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.184821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.193136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.195061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9f608bba-f93a-424b-8eb7-43c7e69f9e04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:04.207972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:04.209147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:04.210025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:04.212457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/916] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:04.220321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.229366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.271772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b1df-3c2b-4fb8-ab1c-d66885537ab4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.272096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b1df-3c2b-4fb8-ab1c-d66885537ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.272356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78b1df-3c2b-4fb8-ab1c-d66885537ab4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.273930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/917] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:04.279560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.279835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.280106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.305394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:04.306493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:04.341537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bcee45-94b2-44d7-8218-45587a622bdb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.379209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.380177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.380659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.381092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.382049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:04.382614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.384331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.391866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/918] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:04.404184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/919] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:04.410543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.414684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.450574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.507918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.525015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2217-1b1d-4dc9-86c4-794229a85b2b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.558163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51bb3-c271-48d9-9ca2-44864ac0ffc8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.562382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.563028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.563370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.563729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.564571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] RBAC: Adding query filter params (role.id=6dce21d292e04e9a82b71e2cd7499510, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=baa77a1e4ec94a159eccbee7db74bcf4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:04.564945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.566441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.579062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/920] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/role_assignments?user.id=baa77a1e4ec94a159eccbee7db74bcf4&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=6dce21d292e04e9a82b71e2cd7499510 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:04.598898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.599872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.600357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.600824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.601614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.603304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:04.613630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:04.614683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:04.615495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:04.617936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/921] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:04.624656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.731471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.736235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcdfa2-c07e-4dad-b3a7-16ecbf204e3e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.771750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.772396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.772675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.772947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.773604 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:04.773892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.775335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.780770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/922] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:04.786546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.891278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab2b65-1f8d-41c0-90fd-de90fc6e8e38 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:04.924363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:04.929508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:04.930193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:04.930472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:04.930744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:04.931404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:04.931703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:04.933132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:04.944865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:04.945302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/923] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:05.048095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b3cf-418a-42a9-af43-3aefca1fb2b9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:05.085309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:05.085926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.086202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.086472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.087362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] RBAC: Adding query filter params (interface=public, service_id=33eec93e3b8448169da5a92c0dc9d792, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:05.087667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:05.089110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8b0e3-9607-4ecb-ad2c-4126432c8033 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:05.094956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/924] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:06:04 2022] GET /identity/v3/endpoints?service_id=33eec93e3b8448169da5a92c0dc9d792&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:05.101939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:05.211243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac39739-c2db-48bb-bc91-ee985ae20ede None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:05.245730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:05.246334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.246608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.246877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.247476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:05.249008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cbea0-5aae-48e3-afbc-d06ceedbc8f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:05.255684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/925] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:05 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:06:05.312663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ebdb-5fcc-4938-bddc-8cc05d099394 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.313028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ebdb-5fcc-4938-bddc-8cc05d099394 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.313212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1ebdb-5fcc-4938-bddc-8cc05d099394 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.315042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/926] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:05.321164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.321500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.321702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.341393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:05.342644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:05.403221 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:05.418411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/927] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:05.441156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1ebdb-5fcc-4938-bddc-8cc05d099394 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:05.470872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:05.553735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1ebdb-5fcc-4938-bddc-8cc05d099394 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:05.588353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:05.589337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.589811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.590273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.591070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:05.592660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:05.602195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:05.603253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 11:06:05.604158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:05.607041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/928] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 9 11:06:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:05.613541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:05.724213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a215-42e9-43b2-9225-ae35083b598a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:05.761769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:05.762835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.763364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.763637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.764336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:05.764644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:05.766240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:05.772834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/929] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:05.778947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:05.879085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:05.889817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729d126-0df6-4397-a71a-5900d2b355e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:05.933906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:05.934842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:05.935332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:05.935797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:05.936713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:05.937203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:05.938787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df6b4f-6c6a-44b7-9891-1c15e1370a31 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:05.944354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/930] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:05.951740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:06.005268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:06.053678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2287e0-3527-4259-a2e6-e9e234a43406 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:06.088934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:06.089540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.089820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.090088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.090757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:06.092277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd574f0-a362-475e-a728-15a63ad5c062 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:06.115089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/931] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:06.241142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62c6-0527-4e2f-815c-39669e9fdfd0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.241675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62c6-0527-4e2f-815c-39669e9fdfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.241908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f62c6-0527-4e2f-815c-39669e9fdfd0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.243655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/932] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:06.249758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.250061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.250310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.270112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:06.271574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:06.351173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/933] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:06.374394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f62c6-0527-4e2f-815c-39669e9fdfd0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:06.482066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f62c6-0527-4e2f-815c-39669e9fdfd0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:06.517489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:06.518102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.518376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.518643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.519299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:06.521273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:06.523979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 11:06:06.544186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 11:06:06.544186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 11:06:06.544186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [parameters: {'id': 'b820f5803b894a4dad94997ba60e95ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 11:06:06.544186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-44c3a8cf-9b0f-4539-bdc5-c1d0322faf4e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b820f5803b894a4dad94997ba60e95ad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b820f5803b894a4dad94997ba60e95ad. Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:06:06.561476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 11:06:06.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Aug 09 11:06:06.565542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 11:06:06.567700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 11:06:06.569776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:06:06.571648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'id': 'b820f5803b894a4dad94997ba60e95ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:06.581563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:06.583373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b820f5803b894a4dad94997ba60e95ad. Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:06.585414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/934] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Aug 9 11:06:06 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:06.586580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:06.606000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:06.680375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe5b08-bf8a-4db7-a4be-9f07648eecb2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:06.714713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ddc3d-03fd-4b1b-91d6-9e602040e9f9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.715034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ddc3d-03fd-4b1b-91d6-9e602040e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.715275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ddc3d-03fd-4b1b-91d6-9e602040e9f9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.716242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:06.716889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.717155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.717413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.723481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/935] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:06.724793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:06.725264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.725545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.725813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.726290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:06.738148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:06.740091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:06.741234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/936] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:06.745883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:06.747364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:06.753381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:06.754028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:06.840483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/937] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:06.859185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:06.862109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:06.870739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3a70e-c7d7-4b78-b304-2197ff384cb6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:06.915543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:06.916430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:06.917123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:06.917464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:06.918203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:06.918567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:06.920089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:06.948310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/938] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:06.985750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8b377-5482-4b23-ba3c-81a67855ccd3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.024869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.025514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.025848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.026119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.035444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.039161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:07.044913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:07.045884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:07.046758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:07.047577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/939] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:07.054416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.163272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d2f46-24a2-47a6-8d6a-b19d6caa2fb4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.202180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.203181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.203707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.204179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.205210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:07.205700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.207290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f5b5-2305-4d78-9aa4-2b10956b8d2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.220390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/940] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:07.223405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.326916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571ddde-a49d-4959-9271-8215eea40304 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.363926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.365009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.365482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.365963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.377044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.378956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e7a9d925-2961-4943-9337-4aef0eccc301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:07.394287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:07.395299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:07.396549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:07.396549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:07.396549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:07.402551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9d1e2-d73d-4f9c-8bcf-4783adeb3cd6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.402551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9d1e2-d73d-4f9c-8bcf-4783adeb3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.402551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9d1e2-d73d-4f9c-8bcf-4783adeb3cd6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.402551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/942] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:07.402923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/942] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:07.405104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9d1e2-d73d-4f9c-8bcf-4783adeb3cd6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.408151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.408461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.408726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.429071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.429361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:07.430497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:07.478030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.498278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/943] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:07.520221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.525651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9d1e2-d73d-4f9c-8bcf-4783adeb3cd6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.565002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.566007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.566480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.567070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.568069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:07.568623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.570292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.584340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/944] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:07.589732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.632401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58151966-ced0-45a1-8254-3dd75b5195d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.670758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.671576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.672764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.673314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.677339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.677339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.677339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:07.685312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:07.686276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:07.687193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:07.690785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/945] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:07.698804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c71fc-ace5-4137-99e8-3dbc10d6e498 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.698963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.731602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.732233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.732554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.732819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.740629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.742069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.746338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:07.754701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:07.755846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:07.756872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:07.760651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/946] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:07.761854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.805528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923863e-ff31-4065-9ff6-e42329ce2794 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.840534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.841446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.841889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.842441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.843543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:07.844129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.845915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.856047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/947] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:07.857286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.866951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8b405-a64f-4ca5-88d3-9999827c2276 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:07.902508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:07.903402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:07.903680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:07.903946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:07.904722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:07.905021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:07.907048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:07.910478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:07.934590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/948] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:07.935872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:07.975881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d60a-fc19-4fd8-a5a6-c0cf754c514a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.013911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.015483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.016020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.016534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.017449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:08.018015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.019581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.030698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.031449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/949] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:08.032794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.055931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4a079-c6c8-4063-bfd6-0b9cd23349b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.096116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.096793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.097068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.097334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.110661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.112578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:08.132057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:08.133056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:08.133940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:08.133940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:08.133940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:08.138137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/950] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:08.139620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.154804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c9f37-9576-4ccc-81a8-a24f6a2d0a76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.197405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.201014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.201892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.202302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.202654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.204006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.205678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b71440-e942-4bbf-b505-74364a1abafc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.236213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/951] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:08.255545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6609-27d1-4d2f-82a0-df98956e9616 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.255875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6609-27d1-4d2f-82a0-df98956e9616 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.256077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6609-27d1-4d2f-82a0-df98956e9616 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.257388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629810-5778-47a9-adeb-2b0b80ee5e02 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.257739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/952] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:08.263676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.263946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.264220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.289903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:08.291619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:08.307662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.308303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.308580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.308845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.309567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:08.309872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.311804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.323473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.331525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/953] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:08.333588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.380366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/954] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:08.398472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.449683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4639-25d2-4173-99a8-741b86f2b29a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.451561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.491974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.492878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.493288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.493692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.501126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.502747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:08.513938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:08.515490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:08.516723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:08.518701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/955] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:08.520998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217e0e1-5699-4711-81cd-b8685ebb6806 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.524486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.563901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.564556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.564823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.565084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.566715 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.568102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:08.578993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:08.580102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:08.581076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/956] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:08.589162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.625016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.651418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1eb52-c468-445b-998b-c586b4c34c80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.693407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.694262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.694539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.694805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.695490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:08.695788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.697652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.713765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3abfc-97e5-41d7-bf1d-7f8545d128a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.713946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/957] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:08.719729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.746443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.758359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.759406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.759899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.760425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.761258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:08.761561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.762958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.773406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/958] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:08.774396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:08.824338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527230d-245f-41ff-8fd6-13039ab093ba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.859351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.860007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.860309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.860582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/baa77a1e4ec94a159eccbee7db74bcf4/roles/6dce21d292e04e9a82b71e2cd7499510` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.862780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=baa77a1e4ec94a159eccbee7db74bcf4, role_id=6dce21d292e04e9a82b71e2cd7499510)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.864232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e002f0-fd45-40b4-a2a1-282deda12b77 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.870466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:08.885841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d91c1-0955-47d6-8896-dbdde17e0df0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:08.886215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/959] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:06:08 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/baa77a1e4ec94a159eccbee7db74bcf4/roles/6dce21d292e04e9a82b71e2cd7499510 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:06:08.926574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:08.927194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:08.927473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:08.927795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:08.928467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:08.928768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:08.930140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dd955-7543-4d25-8450-27a1f9e32c79 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:08.935951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/960] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:09.153466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104d90-dab5-4399-a2fa-1b46cbad501c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:09.153985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104d90-dab5-4399-a2fa-1b46cbad501c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:09.153985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104d90-dab5-4399-a2fa-1b46cbad501c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:09.155611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/961] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:09.161069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:09.161354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:09.161629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:09.180318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:09.181507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:09.250792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/962] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:09.276088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104d90-dab5-4399-a2fa-1b46cbad501c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:09.366493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:09.391450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104d90-dab5-4399-a2fa-1b46cbad501c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:09.429742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:09.430348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:09.430636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:09.430901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:09.431577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:09.433180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:09.435840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING keystone.server.flask.common [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 11:06:09.449167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 11:06:09.449167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 11:06:09.449167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [parameters: {'user_id': '3facd51a5f174273ab757deff16fe5b9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 11:06:09.449167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Aug 09 11:06:09.456062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:06:09.457016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 11:06:09.458385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 11:06:09.459531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 11:06:09.460477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 11:06:09.461426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'user_id': '3facd51a5f174273ab757deff16fe5b9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:09.469275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 11:06:09.471211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:09.473421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/963] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 9 11:06:09 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:09.474321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:09.553351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:09.574550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca368bc-b470-4144-b29e-34a37f73ae51 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:09.610059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:09.611026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:09.611508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:09.612119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:09.619929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:09.621799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-54a7d1ea-79e4-4b04-8a4f-99bd2cab3690 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:09.634291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:09.635264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 11:06:09.636333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:09.639052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/964] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:09.645014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:09.750072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc542b7-e594-441c-afbd-2fdba29eb237 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:09.785544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:09.786150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:09.786424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:09.786691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:09.787388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:09.787686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:09.789352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab66c7-1caa-4d61-8221-c988c4ed965d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:09.827251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/965] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 9 11:06:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:10.323306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961786-2aa1-49e6-8344-b065b2deb60c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.323697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961786-2aa1-49e6-8344-b065b2deb60c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.323827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961786-2aa1-49e6-8344-b065b2deb60c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.325452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/966] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:10.330974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022950a-c484-4ac4-937b-312952a28902 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.331443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022950a-c484-4ac4-937b-312952a28902 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.331521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022950a-c484-4ac4-937b-312952a28902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.338076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:10.351454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d022950a-c484-4ac4-937b-312952a28902 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:10.353816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d022950a-c484-4ac4-937b-312952a28902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:10.426673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/967] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:10.448620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e961786-2aa1-49e6-8344-b065b2deb60c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:10.558449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e961786-2aa1-49e6-8344-b065b2deb60c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:10.600578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:10.601388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.601797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.602164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.603003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:10.605255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:10.617024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:10.618070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:10.618926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:10.620439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/968] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:10.626735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022950a-c484-4ac4-937b-312952a28902 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:10.731732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022950a-c484-4ac4-937b-312952a28902 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:10.760825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:10.767276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:10.767882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.768147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.768442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.769064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:10.769354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:10.771058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b501c71-a125-494d-83a7-d90ea42d2c3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:10.776085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/969] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:10.781740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:10.791451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:10.881782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c16e3-d28d-489a-809e-bb9cf8adaef9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:10.922600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:10.923279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.923551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.923814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.924551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:10.924891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:10.926318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:10.936686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/970] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:10.937801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c13f6-2f34-4fa7-b6f4-c01ffff30ce3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.938002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:10.938100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c13f6-2f34-4fa7-b6f4-c01ffff30ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.938334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c13f6-2f34-4fa7-b6f4-c01ffff30ce3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.940122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/971] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:10.945023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90d3f-b299-4dfd-b80b-08e6122ad730 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:10.945304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90d3f-b299-4dfd-b80b-08e6122ad730 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:10.945570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90d3f-b299-4dfd-b80b-08e6122ad730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:10.965944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4ac90d3f-b299-4dfd-b80b-08e6122ad730 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:10.967124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ac90d3f-b299-4dfd-b80b-08e6122ad730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:11.027030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.039996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/972] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:11.041287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bec2-86ff-4f1e-b1e5-ec56c55ef811 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.041566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bec2-86ff-4f1e-b1e5-ec56c55ef811 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.041835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bec2-86ff-4f1e-b1e5-ec56c55ef811 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.043051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/973] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:11.048527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.048805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.049077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.062626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474534-1453-46a2-b9f0-fdbb829dcdd2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.067338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:11.068556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:11.097282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.097880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.098149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.098415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.099108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] RBAC: Adding query filter params (interface=admin, service_id=33eec93e3b8448169da5a92c0dc9d792, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:11.099409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.100784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.105594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/974] 10.222.0.27 () {56 vars in 1249 bytes} [Tue Aug 9 11:06:10 2022] GET /identity/v3/endpoints?service_id=33eec93e3b8448169da5a92c0dc9d792&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:11.106348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.137246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/975] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:11.137884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.194777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.207387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0086b-d4bd-4675-b410-c0fe5f85bc40 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.241266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f5387-bdb8-448e-9751-927eeb03f5ed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.241902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.242525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.242800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.243064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.245191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.255927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:11.260643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:11.261708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/976] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:11.262545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.282668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.283279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.283552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.283817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.284450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.285818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.292008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/977] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:06:11.292843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.389836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39587c-f069-4900-bd3c-fa14c07249eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.407685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277ee3b-2715-4308-9065-126aa92f4bf3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.432104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.432753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.433031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.433302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.433957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.435331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.438231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.460293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/978] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:06:11 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:11.465524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.466156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.466432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.466700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.467387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:11.467723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.469537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.479169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/979] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:11.485675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.548589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.601165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d040d9-f486-4c1c-8111-00638f9e9145 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.641129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.641713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.641982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.642241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.654048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.655547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:11.670678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:11.672405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:11.673336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:11.673336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:11.673336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:11.673336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/980] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:11.677779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.783739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d24a-4e14-433a-b27b-1c3730ffdc70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.820111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.820762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.821033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.821292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.821978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:11.822276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.823638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:11.836367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/981] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:11.841798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:11.948412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c771634-5b97-4e41-b148-ce3f5cd06d72 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:11.956698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:11.984897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:11.985516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:11.985809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:11.986121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:11.993830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:11.995242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:12.006619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:12.007605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:12.013444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/982] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:06:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:12.014066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:12.056066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:12.117166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c38257-752b-4f9c-b521-15948443d328 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:12.151546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:12.152136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:12.152443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:12.152700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:12.153414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:12.153707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:12.155173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:12.173464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/983] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 9 11:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:12.178865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:12.285650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5069cc-3d1a-456c-9745-57a6df9e70bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:12.330925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:12.331804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:12.332204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:12.332631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:12.368061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:12.403047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:12.404508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:12.505249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/984] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:12 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:12.510761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:12.614588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:12.622484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e256e-64e7-4eda-9591-ff4ae9d5b4d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:12.660689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:12.661321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:12.661596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:12.661860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:12.672724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:12.674196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:12.688761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:12.689686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:12.695367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:12.695367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:12.695367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:12.695367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/985] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:12.695603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:12.761519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:12.796310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccb3b0-9e7b-4505-b7e8-d6230df1585f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:12.829188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:12.829798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:12.830073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:12.830340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:12.830975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:12.831272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:12.832694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:12.844252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/986] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:12.849755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:12.960734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb838a-2c36-43ef-a3ba-cdf3d423c34b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:12.996008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:12.996654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:12.996922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:12.997181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.004237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.005698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:13.016994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:13.017869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:13.023837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/987] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:13.024383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.126926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febd2da-1e1c-418e-b3a9-e1f220c9e147 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.152570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.163200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.163812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.164085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.164380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.165079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:13.165377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.166816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d17f80-fb2d-4b71-96e4-1015d6ca53ed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.179411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/988] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:13.185297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.197139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.291448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2a92-1c8e-4203-8b17-1274241c3ecb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.299722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440f09-52db-4621-9342-693cdf108057 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.300009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440f09-52db-4621-9342-693cdf108057 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.300255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440f09-52db-4621-9342-693cdf108057 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.301808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/989] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:13.306660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae486-292a-43ee-ae8f-468b8b7fd455 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.306941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae486-292a-43ee-ae8f-468b8b7fd455 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.307212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ae486-292a-43ee-ae8f-468b8b7fd455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.324830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-971ae486-292a-43ee-ae8f-468b8b7fd455 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:13.326007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-971ae486-292a-43ee-ae8f-468b8b7fd455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:13.328445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.329854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.330131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.330401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.331112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] RBAC: Adding query filter params (role.id=6dce21d292e04e9a82b71e2cd7499510, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=baa77a1e4ec94a159eccbee7db74bcf4) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:13.331418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.332885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.340301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/990] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/role_assignments?user.id=baa77a1e4ec94a159eccbee7db74bcf4&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=6dce21d292e04e9a82b71e2cd7499510 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:13.413101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/991] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:13.436147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.478824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290613-abf8-43b6-b35d-821b2d047612 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.479452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290613-abf8-43b6-b35d-821b2d047612 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.479914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31290613-abf8-43b6-b35d-821b2d047612 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.481574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/992] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:13.487910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.488800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.489265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.508558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:13.509971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:13.519769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.558750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adca137-80eb-40b1-9917-1b32564be780 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.587381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/993] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:13.595114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.595742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.596010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.596291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.596863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.598217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:13.606776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:13.607776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:13.608750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/994] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:13.611950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.614054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.662181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.715946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b146b5c-fd2b-4f0b-972b-0954014c3fbe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.722762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a355e02-d5d5-4076-b5b9-b9f76106cbd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.751265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.751886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.752161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.752479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.753059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.754410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.758361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.759018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.759349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.759716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.760420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:13.760772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.762167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:13.762860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:13.764008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:13.765537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/995] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:13.768105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.769850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.778017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/996] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:13.779185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.884401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04160a-073f-4be1-be90-916e436aa28b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.887218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b83dfb-a971-493d-b944-18e79859247c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:13.895383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:13.926777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.927382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.927649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.927916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.928660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:13.928956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.930399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.931703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:13.932437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:13.932770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:13.933162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:13.933845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:13.934202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:13.935592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:13.936835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/997] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:13.942203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:13.945241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/998] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:13.951105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:14.016443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:14.056975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e2256-ad35-4f20-91bc-ce15cec99895 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:14.057190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eb6da-8870-45ba-988e-81a9d8672dcf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:14.091762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:14.091981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:14.092674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:14.092806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:14.092955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:14.093221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:14.093444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:14.093844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:14.093939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:14.094141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:14.094758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:14.095533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:14.096329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:14.105295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/999] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:14.106158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:14.128300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/1000] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:06:14.160539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:14.226404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8beb5c-f16d-4102-9117-5821824c140f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:14.269612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:14.270218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:14.270810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:14.271983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:14.271983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] RBAC: Adding query filter params (interface=public, service_id=01dea9b5af664f7bba859eee153be823, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:14.272168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:14.273429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0a3a6-c6e4-49fa-9bfb-bdd443e66fcf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:14.278771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1001] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:06:14 2022] GET /identity/v3/endpoints?service_id=01dea9b5af664f7bba859eee153be823&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:14.284489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:14.388499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09bc5-97c0-4bbd-8ead-9d9456104d38 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:14.412338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:14.424937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:14.425551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:14.425824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:14.426070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:14.426691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:14.428243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d519b-b1be-4689-aa3f-cc88150de569 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:14.434795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/1002] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:14 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:06:15.366171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b022-c576-4d88-a667-5914b79e8d5f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:15.366752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b022-c576-4d88-a667-5914b79e8d5f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:15.366752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b022-c576-4d88-a667-5914b79e8d5f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:15.371552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1003] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:15.373210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:15.373632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:15.373732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:15.391957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:15.393154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:15.464323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:15.485189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926b022-c576-4d88-a667-5914b79e8d5f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:15.589539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:15.595483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926b022-c576-4d88-a667-5914b79e8d5f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:15.628953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:15.629492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:15.629765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:15.630027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:15.630596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:15.632105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:15.641234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:15.642218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:15.643059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:15.647707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1005] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 9 11:06:15 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:15.649027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:15.754906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914d3b1-1a38-48f8-85fa-287c941fbdb8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:15.773325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:15.791562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:15.792175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:15.792475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:15.792743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:15.793375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:15.793672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:15.795036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e7c5b-f911-4b9f-bd0d-9c0b22454e46 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:15.804506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/1006] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:15.805653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:15.914386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4402-d452-47a8-b047-9c36858b68b5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:15.950642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:15.951241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:15.951514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:15.951775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:15.952505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:15.952792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:15.954138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0719b3-601f-4fe0-97d1-1da9c46c63e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:15.964513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1007] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:16.140744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bc8d-854a-423f-b8ab-3eb4df277636 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.141029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bc8d-854a-423f-b8ab-3eb4df277636 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.141289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907bc8d-854a-423f-b8ab-3eb4df277636 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.142654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/1008] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:16.148678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644740d-ef25-4ff4-8c1a-09e82a4a9133 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.148958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644740d-ef25-4ff4-8c1a-09e82a4a9133 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.149222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644740d-ef25-4ff4-8c1a-09e82a4a9133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.168502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5644740d-ef25-4ff4-8c1a-09e82a4a9133 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:16.169600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5644740d-ef25-4ff4-8c1a-09e82a4a9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:16.192521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:16.241605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1009] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:16.264180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907bc8d-854a-423f-b8ab-3eb4df277636 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:16.374054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907bc8d-854a-423f-b8ab-3eb4df277636 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:16.411377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:16.411377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.411377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.411377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.412645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:16.414083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:16.415473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050ebf-402e-4a78-a097-a74f5883cf3a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.415883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050ebf-402e-4a78-a097-a74f5883cf3a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.415966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/1010] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:16.416038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c050ebf-402e-4a78-a097-a74f5883cf3a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.418668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1011] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:16.422076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:16.422594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.422855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.423131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.440881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:16.442107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:16.474120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:16.512324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1012] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:16.525869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3222c7-78bb-43a4-949e-e25027d200eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:16.530688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:16.566140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:16.567261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.567734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.568210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.569221 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:16.570853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:16.599231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/1013] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:06:16 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:06:16.631563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:16.652532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba865c1e-403f-4ddd-ab31-df7c870d7717 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:16.693050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:16.694110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.694574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.695105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.695877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:16.697526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:16.708300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:16.719042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:16.721374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1014] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:16.723586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:16.830624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926c7f1-9c96-4ea1-8b03-30642b73a9e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:16.832208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:16.866446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:16.867014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:16.867280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:16.867539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:16.868186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:16.868511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:16.869861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:16.874986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:16.880547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:16.989599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f5fba-4dc6-4793-941d-e5628fedefcc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:17.025842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:17.026764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:17.027218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:17.027675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:17.028572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:17.028874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:17.030296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afacb4d-4247-4ff6-9176-35b5ef936339 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:17.036565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1016] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 9 11:06:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:17.041966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:17.150099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf964-924c-4da5-8c42-5e4e43443cf6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:17.182899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:17.183501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:17.183777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:17.184043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:17.184747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:17.186116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c3988-27d2-428e-9ec6-ab12bbfa95d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:17.212299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/1017] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:17.964306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768936c-da59-471f-bd52-3687cd86f326 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:17.964837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768936c-da59-471f-bd52-3687cd86f326 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:17.964837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768936c-da59-471f-bd52-3687cd86f326 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:17.966350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1018] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:17.971439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea29da-c81c-48f2-ac95-92fbb9907205 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:17.971864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea29da-c81c-48f2-ac95-92fbb9907205 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:17.971937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea29da-c81c-48f2-ac95-92fbb9907205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:17.993639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9fea29da-c81c-48f2-ac95-92fbb9907205 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:17.994811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fea29da-c81c-48f2-ac95-92fbb9907205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:18.012600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:18.073869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:17 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:18.096312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768936c-da59-471f-bd52-3687cd86f326 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:18.114034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:18.201731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768936c-da59-471f-bd52-3687cd86f326 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:18.235631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fa466-af01-476d-831e-0865ee687530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:18.236229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa466-af01-476d-831e-0865ee687530 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:18.236528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa466-af01-476d-831e-0865ee687530 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:18.236790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa466-af01-476d-831e-0865ee687530 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:18.237426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fa466-af01-476d-831e-0865ee687530 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:18.238777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fa466-af01-476d-831e-0865ee687530 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:18.254344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:06:18 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:18.682268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bc8b-cbae-4c2a-85cd-00c38e8b384c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:18.682637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bc8b-cbae-4c2a-85cd-00c38e8b384c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:18.682867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680bc8b-cbae-4c2a-85cd-00c38e8b384c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:18.684421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/1021] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:18.690180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:18.690470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:18.690807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:18.709857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:18.711045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:18.784462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:18.790709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:18.814460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680bc8b-cbae-4c2a-85cd-00c38e8b384c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:18.935811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680bc8b-cbae-4c2a-85cd-00c38e8b384c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:18.976105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:18.977006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:18.977087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:18.977328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:18.978805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:18.980309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55939-219f-47d4-b37f-ca04da798214 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:18.987110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/1023] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:18.988518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:19.101711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa36d55-1c0c-4eff-b6ec-96d0bc2b4325 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:19.139800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:19.140791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.141313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.141949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.142306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:19.143767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff91b4-a5d5-43a1-ae5a-6520ddb3e324 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:19.174586 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1024] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:06:18 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:19.352144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975e07a-6ee2-4367-8c52-4297b004f68f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.352546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975e07a-6ee2-4367-8c52-4297b004f68f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.352740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975e07a-6ee2-4367-8c52-4297b004f68f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.354354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/1025] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:19.359542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.359889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.360225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.378873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:19.380027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:19.435316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:19.454732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1026] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:19.477786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975e07a-6ee2-4367-8c52-4297b004f68f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:19.497046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:19.586306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975e07a-6ee2-4367-8c52-4297b004f68f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:19.621972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:19.622520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.622793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.623057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.634158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:19.639633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:19.640582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:19.641421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/1027] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:19.649132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:19.750640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58457476-be66-4b3b-b796-33694b2d33c1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:19.785008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:19.785928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.786382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.786827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.787660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:19.788139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:19.789696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:19.802412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1028] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:19.804232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:19.886350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:19.915819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ab74c-7b85-4280-b0fc-0f30c93dfa6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:19.952184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:19.953128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:19.953549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:19.953994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:19.964788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:19.966471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:19.981736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:19.983426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:19.989233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:19.989233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:19.989233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:19.989233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1029] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:19.991041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.043349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.096986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf53-3d0b-498b-b149-be935e9c3432 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.130956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.131552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.131822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.132086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.132753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:20.133049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.134392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.148724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1030] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:20.154112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.255790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96029051-d409-4201-bbdd-1ca054a4beb4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.282979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.292298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.292918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.293188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.293455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.301574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.303009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6c1a2439-40a3-46a2-8e5e-772bcab9e11d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:20.314380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:20.315344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:20.316848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:20.320654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1031] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:20.321983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.330911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbc86-4939-46a4-bb1e-6304753d2c16 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.331256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbc86-4939-46a4-bb1e-6304753d2c16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.331519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbc86-4939-46a4-bb1e-6304753d2c16 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.332983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1032] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:20.338775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.339206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.339311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.358180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:20.359915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:20.429780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1033] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:20.433765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76be74-3c7e-41fb-9e70-ba3be15d9c48 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.443412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.452164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.475835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.476492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.476744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.477016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.477739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:20.478037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.479454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.497188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1034] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:20.503104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.560671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bd8b9-123d-44e6-82ed-104eaab0ca31 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.580842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.599474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.600096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.600392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.600655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.601227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.602582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.608231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c588d-07d5-4d0b-8a4d-a8de0e4e7d03 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:20.611262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:20.612747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:20.613785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1035] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:20.618165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.651263 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.651857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.652130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.652426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.687576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.721873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8010a9-8cd7-40f4-880d-18a82d0bc7f7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.723159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.724635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.760396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.761008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.761288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.761554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.762184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:20.762488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.763933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.775117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1036] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:20.777027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.828663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.835912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1037] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 11:06:20.841902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.888061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dbc73-6b79-454a-8ec6-17153aaf659a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.914579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:20.928407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.929024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.929296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.929558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:20.930177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:20.930470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:20.931830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:20.937270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1038] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:20.943007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:20.956557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94596cd-8698-4943-b39d-6e3e28fe0cd8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:20.998236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:20.998961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:20.999242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:20.999542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.009764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.011192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:21.026331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:21.027328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:21.028227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:21.028227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:21.028227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:21.028227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1039] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:21.033161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.050771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7df705-a95e-4b43-b301-f62e55c2fb14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.061981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.088194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.088920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.089202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.089461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.090131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] RBAC: Adding query filter params (interface=public, service_id=e1f1756ba0134b379262afb1419d727b, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:21.090447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.092000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.097697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1040] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:06:20 2022] GET /identity/v3/endpoints?service_id=e1f1756ba0134b379262afb1419d727b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:21.103253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.142838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8ca7-d73f-4660-aed8-f9d657b005a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.163272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.181231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.181838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.182113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.182383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.183026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:21.183326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.184726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.198332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1041] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:21.203941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.223780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bf258-5156-4cc0-aa9b-994f24fb6ad3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.260787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.261401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.261673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.261939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.262540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.263997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d7414-98db-45d8-95be-2cb30fd94966 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.271805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1042] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:06:21.285509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a6ef-2c51-411b-b444-08fca10eaa8f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.285788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a6ef-2c51-411b-b444-08fca10eaa8f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.286059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a6ef-2c51-411b-b444-08fca10eaa8f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.287678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1043] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:21.293469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.293800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.294017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.314626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:21.315837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:21.321876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca9f7-0aeb-4790-baf2-07e3f67bda2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.349380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.363878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.364559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.364837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.365109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.372864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.374443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:21.388340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:21.389913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:21.391097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1044] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:21.396240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.404693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1045] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:21.428610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.442075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.512468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626262c3-13a6-4218-be58-8a94ebebddd7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.541034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9e256-8884-497a-88ae-16a61858f062 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.547853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.548501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.548773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.549034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.549712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:21.550009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.551420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.564905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1046] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:21.571291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.579200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.580180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.580714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.581147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.592667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:21.598016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:21.599103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:21.600156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:21.602311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1047] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:21.608428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.648225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.677262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:21.681590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb0662-911e-474d-8568-c666893ad54e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.717746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.718335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.718602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.718859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.719568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] RBAC: Adding query filter params (role.id=6dce21d292e04e9a82b71e2cd7499510, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:21.719914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.721418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.725327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd13b8-c7ea-4357-afd1-7440b867b4c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.732522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1048] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=6dce21d292e04e9a82b71e2cd7499510 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:21.769259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.769794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.770071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.770331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.771323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:21.771617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.773065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.788706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:21.789781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:21.899071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8aedfb-a450-4f12-ab51-0f3ff469dd91 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:21.938730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b71-198d-458d-9706-081c196954ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:21.939351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b71-198d-458d-9706-081c196954ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:21.939621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b71-198d-458d-9706-081c196954ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:21.939884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b788b71-198d-458d-9706-081c196954ef None admin] PATH_INFO: `/v3/users/baea138a928240d180add2b40965f1ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:21.953803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788b71-198d-458d-9706-081c196954ef None admin] RBAC: Authorizing `identity:get_user(user_id=baea138a928240d180add2b40965f1ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:21.955206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b788b71-198d-458d-9706-081c196954ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:21.957021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1050] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/users/baea138a928240d180add2b40965f1ac => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:21.962605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:22.055972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:22.066800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ca151-3647-414c-be19-56468cf41050 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:22.100421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:22.101017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:22.101285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:22.101545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:22.168902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:22.168902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:22.255549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:22.267100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1051] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:22.273152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b71-198d-458d-9706-081c196954ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:22.294943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:22.373904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b788b71-198d-458d-9706-081c196954ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:22.410259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:22.410876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:22.411144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:22.411406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:22.412557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:22.413970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18714871-9338-45ff-a3e7-f0b90be5a515 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:22.415803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1052] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:22 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:22.421868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:22.518994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b5ea7-ff91-4b86-b1e5-cbcb89af1985 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:22.552211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebad29-6323-46a0-8449-87025644200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:22.552826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebad29-6323-46a0-8449-87025644200b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:22.553091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebad29-6323-46a0-8449-87025644200b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:22.553354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebad29-6323-46a0-8449-87025644200b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:22.554055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebad29-6323-46a0-8449-87025644200b None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=18a806e76e2949d6a72dc62f9875785c, user.id=baea138a928240d180add2b40965f1ac) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:22.554358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebad29-6323-46a0-8449-87025644200b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:22.555765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebad29-6323-46a0-8449-87025644200b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:22.561220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1053] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:22 2022] GET /identity/v3/role_assignments?user.id=baea138a928240d180add2b40965f1ac&scope.project.id=18a806e76e2949d6a72dc62f9875785c&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:23.276998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622876-f481-47c2-a7f2-dbc6524d654e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.277540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622876-f481-47c2-a7f2-dbc6524d654e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.277540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622876-f481-47c2-a7f2-dbc6524d654e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.278917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1054] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:23.284983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.285463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.285463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.303972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:23.305170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:23.355455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:23.374689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:23.398308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9622876-f481-47c2-a7f2-dbc6524d654e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:23.489652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:23.503653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9622876-f481-47c2-a7f2-dbc6524d654e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:23.537990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:23.538634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.538764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.539028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.539599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:23.540966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0176cff5-69ad-4ba2-841f-9ddce517595b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:23.549637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:23.550610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:23.556581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1056] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 9 11:06:23 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:23.557402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:23.658915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5265c-c9d8-44d8-8ebb-131ac413e215 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:23.693717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:23.694313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.694577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.694836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.695433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:23.695722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:23.697106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:23.701413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4eb0f-3863-4672-ba9a-1ed39cc65133 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.701992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4eb0f-3863-4672-ba9a-1ed39cc65133 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.702271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4eb0f-3863-4672-ba9a-1ed39cc65133 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.702515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1057] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:23.704063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1058] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:23.708530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:23.708796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.709185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.709557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.727765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:23.728983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:23.757422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:23.804816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1059] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:23.812027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cce681-af8b-40fb-8208-7c62c9b48e3a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:23.826634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:23.848206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:23.849059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.849578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.849842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.850464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:23.850762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:23.852122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:23.856888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1060] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:23.862523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:23.914697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:23.937156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23e866-fea9-4420-be58-48ed0b14fda4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:23.971755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:23.972409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:23.972677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:23.972938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:23.975584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46024868-71d4-4a2a-9f7f-44790e1e5331 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:23.983238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:23.987782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:23.988721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:23.989571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:23.992614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:23.994268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1061] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:23.994773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.017391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.018004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.018279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.018546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.019195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.020919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.043550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1062] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:06:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:06:24.104188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65d44-62bd-4c96-a8d9-a33cb3b28ec9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.147792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.148420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.148688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.148944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.149622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:24.149915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.151269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.157338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:24.161897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1063] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:24.167641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.271853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0311-a40f-407a-b429-1da36d3c1b17 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.306660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.307332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.307580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.307794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.318706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.320139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:24.333836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:24.334776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:24.335661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:24.335661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:24.335661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:24.335661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:24.340882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.443177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2d16-5330-4f5d-b50d-e8b6471ee3c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.476745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.477341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.477608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.477866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.478502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:24.478796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.480144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b9d7a-ee03-4c8d-bc82-e791f3f57af4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.492331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:24.497750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.520087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:24.529930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946a56-c151-4d87-a00e-ee75f837ba26 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.530204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946a56-c151-4d87-a00e-ee75f837ba26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.530465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a946a56-c151-4d87-a00e-ee75f837ba26 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.531761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1066] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:24.538654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.539126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.539401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.558976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:24.560150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:24.612921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a430791-d532-438c-9b10-5b57636ed540 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.618934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:24.631927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1067] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:24.651092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.651715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.651989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.652255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.653691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.660399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.661838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:24.674688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:24.675639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:24.676610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1068] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:24.681623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.767522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649b4e6-b61e-4cd2-b92a-33f0deed6569 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.769478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:24.791559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371787-38d1-4e2d-af8e-184dce6b3ec7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.805729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.806351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.806625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.806890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.817332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:24.821874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:24.822843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:24.824418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1069] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:24.829043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.830433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.831037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.831312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.831575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.832317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:24.832618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.834055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.851667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1070] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:24.857238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.882501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:24.942129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f7c3b-5a28-4921-8a2b-5ae2c36f4b2f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.961350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0584-8412-40d3-9313-e018b82395cd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:24.976863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.977458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.977731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.977995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:24.978650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:24.978943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:24.980316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:24.989451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1071] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:24.995194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:24.997400 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:24.998093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:24.998427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:24.998837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.006292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.010370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.012518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:25.027998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:25.032427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:25.033567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:25.033567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:25.033567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:25.033567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1072] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:25.034868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.105286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034fe43-6861-4546-9e25-feaa1f833917 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.119835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.135297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1637-961b-4d4f-87e9-ee0c72bba4e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.143116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.143741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.144016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.144726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] PATH_INFO: `/v3/users/baea138a928240d180add2b40965f1ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.145833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] RBAC: Authorizing `identity:get_user(user_id=baea138a928240d180add2b40965f1ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.147334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.153714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1073] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:24 2022] GET /identity/v3/users/baea138a928240d180add2b40965f1ac => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:25.154587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.177491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.178112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.178391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.178657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.179318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:25.179616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.181169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.200586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1074] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:25.203047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.253900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.265138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19d3ea-c867-447c-bc2e-e2de906dd825 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.299182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.299728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.300006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.300305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.301477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.303004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.304701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1075] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:25 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:25.310395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.313607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7bf1b-21a1-4fd0-a7af-76bbe67f8bfa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.350677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.351647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.351935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.352203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.359084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.359521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.360942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:25.372176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:25.373199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:25.374183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1076] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:25.379060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.416951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c08e7-c5cb-45b8-b04c-35140916b1ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.452764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.453417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.453697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.453967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c/users/baea138a928240d180add2b40965f1ac/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.455856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=18a806e76e2949d6a72dc62f9875785c, user_id=baea138a928240d180add2b40965f1ac, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.457300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.473207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1077] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:06:25 2022] PUT /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c/users/baea138a928240d180add2b40965f1ac/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:06:25.485853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac59792-61ae-4f58-a257-e4fe4ba05db4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.499045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.528361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.528991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.529319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.529593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.530350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:25.530660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.532148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702c20d-03ab-4dfb-8203-d25cfe3f0912 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.547177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1078] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:25.552522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:25.665842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4070159-6cbc-49e5-a364-ea1c1ebc6ad5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:25.676204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:25.702875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:25.703643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:25.703925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:25.704197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/6dce21d292e04e9a82b71e2cd7499510` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:25.706357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=fbbd789ff1f244eb98d2e5dd001a81c1, role_id=6dce21d292e04e9a82b71e2cd7499510)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:25.707794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ce43-1868-4aea-9a4c-2528d36eaec0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:25.722880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1079] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:06:25 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/fbbd789ff1f244eb98d2e5dd001a81c1/roles/6dce21d292e04e9a82b71e2cd7499510 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:06:26.082011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5089-6378-46b6-80ac-9e9ecbd3125d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:26.082413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5089-6378-46b6-80ac-9e9ecbd3125d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:26.082534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5089-6378-46b6-80ac-9e9ecbd3125d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:26.084057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1080] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:06:26.089174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:26.089549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:26.089889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:26.109237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:26.110455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:26.188393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1081] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:26.207453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d5089-6378-46b6-80ac-9e9ecbd3125d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:26.310622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d5089-6378-46b6-80ac-9e9ecbd3125d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:26.344652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:26.345302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:26.345539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:26.345785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:26.386722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:26.419143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:26.420819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:26.525329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 11:06:26.531488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:26.631748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:26.636502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7fc92-0339-4d15-831d-3f0eae5eb751 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:26.672936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:26.673615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:26.673890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:26.674157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:26.684301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:26.685756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ecfd86c3-d58f-4511-b64d-98cb284361b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:26.701304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:26.702260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:26.703139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:26.703139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:26.703139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:26.707642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:26.709046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:26.778473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:26.810707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902272b8-cf04-4b5b-bd76-5cac8e515c06 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:26.847483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:26.848460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:26.848915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:26.849372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:26.850238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:26.850710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:26.852288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c6f8f-e8db-45db-82b4-0519e31a8aba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:26.871319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1084] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:27.472336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033c3d-a99c-4c1b-9394-609ba7e99bbd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.472619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033c3d-a99c-4c1b-9394-609ba7e99bbd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.472887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033c3d-a99c-4c1b-9394-609ba7e99bbd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.474233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1085] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:27.479385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a786-bf19-4a35-81b5-9323c506cf7b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.479863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a786-bf19-4a35-81b5-9323c506cf7b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.479863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863a786-bf19-4a35-81b5-9323c506cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.498406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e863a786-bf19-4a35-81b5-9323c506cf7b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:27.499718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e863a786-bf19-4a35-81b5-9323c506cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:27.567564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:27.589948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f033c3d-a99c-4c1b-9394-609ba7e99bbd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:27.695575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f033c3d-a99c-4c1b-9394-609ba7e99bbd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:27.710206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f74b2-721f-4b8e-acef-be10e552276e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.710502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f74b2-721f-4b8e-acef-be10e552276e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.710768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f74b2-721f-4b8e-acef-be10e552276e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.712070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1087] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:27.717004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.717310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.717575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.730697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:27.731240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.731510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.731774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.736410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:27.737596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:27.742612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:27.747167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:27.748199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:27.749179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1088] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:27.754388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:27.766753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:27.798672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:27.811680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1089] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:06:27.835507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:27.860939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49603451-2366-4534-a534-d66ac9869c69 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:27.894196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:27.894789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.895065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.895652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.896337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:27.896640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:27.898002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:27.906202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1090] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:27.912110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:27.938472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb598031-10a8-412b-98f4-f1e36a9256f2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:27.971325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:27.971941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:27.972210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:27.972497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:27.982304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:27.985890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:27.986934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:27.988010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:27.989093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1091] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 9 11:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:27.995687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.019856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911013be-48ec-40d1-b4f1-ea99e7b59933 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.047092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.056513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.057112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.057381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.057641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] PATH_INFO: `/v3/users/baea138a928240d180add2b40965f1ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.058766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=baea138a928240d180add2b40965f1ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.060179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.061859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1092] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:27 2022] GET /identity/v3/users/baea138a928240d180add2b40965f1ac => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:28.067589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.099102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a116-b5e7-4e4d-8022-5ef9093fce92 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.132436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.133031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.133299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.133561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.134213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:28.134504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.135860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.143951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1093] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:28.150069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.165595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd6282-8187-49f3-957f-da2a3fa80ae1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.199066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.199660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.199951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.200207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.210382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.248772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49ef15-7ea2-4104-975e-0c5c8697e80e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.261576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.268007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.269494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.284124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.284781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.285053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.285317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.295965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.297614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:28.311337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:28.312496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:28.313377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:28.313377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:28.313377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:28.313377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:06:28.318200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.370727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1095] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 11:06:28.376801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.416096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.424729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a414b5-0fcb-4132-b6e7-f1c0e93d3cef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.457737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.458337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.458636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.458898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.459541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:28.459818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.461222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.473449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1096] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:28.473899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e556f8-48c2-45d5-903f-e68ba0b77ef2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.479311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.503600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.507375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.508016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.508321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.508594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.509697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.511082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.512902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1097] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:28.518789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.577347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561f57c-09f8-4446-b289-d96bb8ea81af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.610470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.611084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.611357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.611613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.614769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc940707-29a0-4a05-aab5-0668a28a703e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.618523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.619943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.623833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:06:28.631372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:28.632527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:28.633526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1098] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:28.638734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.651442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.652107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.652411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.652654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.653386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=18a806e76e2949d6a72dc62f9875785c, user.id=baea138a928240d180add2b40965f1ac) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:28.653690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.655253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.660979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1099] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/role_assignments?user.id=baea138a928240d180add2b40965f1ac&scope.project.id=18a806e76e2949d6a72dc62f9875785c&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:28.731121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:28.749256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb1e9f-d678-44d3-934d-293e0c3a2806 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.786823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.787333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.787607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.787872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.788642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:28.788937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:28.790384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:28.809447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1100] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:28.815695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:28.919599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8830c-12ae-491a-aa16-3dcd79252a51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:28.956251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:28.956949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:28.957220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:28.957483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:28.998375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:29.026015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.027473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.128806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1101] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:29.134806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:29.242941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d26bf-12e1-44a2-be55-01694e633361 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:29.277445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:29.278060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:29.278328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:29.278590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:29.288851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.290297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.292053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:29.306035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:29.307038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:06:29.307922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:29.307922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:06:29.307922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:29.307922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1102] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:29.313145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:29.383383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:29.415551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b756c-e21e-47b3-b3e3-95e1af36d6fe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:29.450253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:29.450952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:29.451223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:29.451633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:29.452613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:29.452913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.454262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.465073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1103] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:29.471165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:29.576502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d37db4-2540-4122-849a-923f381e7da1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:29.610258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:29.610864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:29.611130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:29.611391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:29.618062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.619481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:06:29.631364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:29.632284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:29.633184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1104] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:29.639184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:29.739569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c44b3-87e9-4aac-9114-297ca85bfae8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:29.766920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:29.772626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:29.773221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:29.773490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:29.773754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:29.774432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:29.774727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.776139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447feec-28d2-4119-9337-090f7168d343 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.789338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:29.795682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:29.850749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:29.896758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7133df-73dc-4238-b296-793c2c8a5e86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:29.931017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:29.931620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:29.931892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:29.932154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:29.932895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] RBAC: Adding query filter params (role.id=6dce21d292e04e9a82b71e2cd7499510, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=fbbd789ff1f244eb98d2e5dd001a81c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:29.933194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:29.934607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a737e-dd67-4550-9c80-bcce3588cea7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:29.940436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:29 2022] GET /identity/v3/role_assignments?user.id=fbbd789ff1f244eb98d2e5dd001a81c1&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=6dce21d292e04e9a82b71e2cd7499510 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:30.687113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9e53-618a-4b36-aa1c-511de25b3045 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:30.687661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9e53-618a-4b36-aa1c-511de25b3045 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:30.687661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9e53-618a-4b36-aa1c-511de25b3045 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:30.689284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1107] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:30.697265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:30.697711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:30.697816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:30.722579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:30.723472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:30.795107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:30.819467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec9e53-618a-4b36-aa1c-511de25b3045 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:30.926113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec9e53-618a-4b36-aa1c-511de25b3045 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:30.966666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:30.967262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:30.967530 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:30.967794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:30.968902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:30.970310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5dcbb-15c6-4836-b126-2c8fd0904ffe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:30.971845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:30.977807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:31.030128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:31.086872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0db94-aaac-4823-b8e4-7f9d0b474d25 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:31.122460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:31.123050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:31.123294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:31.123643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:31.124305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:31.125719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444275e-37c7-4b51-80a5-adc4026fbaca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:31.150839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:06:30 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:33.168345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280cc9-6680-48f6-81c4-1785317fdcca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:33.169156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280cc9-6680-48f6-81c4-1785317fdcca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:33.169156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280cc9-6680-48f6-81c4-1785317fdcca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:33.171007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1111] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:33.176394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:33.176839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:33.176930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:33.201893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:33.203088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:33.279141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:33.303368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280cc9-6680-48f6-81c4-1785317fdcca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:33.368551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:33.412599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280cc9-6680-48f6-81c4-1785317fdcca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:33.448647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:33.449186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:33.449456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:33.449717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] PATH_INFO: `/v3/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:33.451209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] RBAC: Authorizing `identity:get_role(role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:33.452649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:33.453937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:33 2022] GET /identity/v3/roles/7f08073ba42c48ff8060c007983602f6 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:33.459945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:33.500969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:33.581111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed10eb-1450-4dad-a9b8-f1420b309e6e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:33.615306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:33.615921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:33.616191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:33.616490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] PATH_INFO: `/v3/users/0fa271b4851840cd88706fa2e3aa83f2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:33.630793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=0fa271b4851840cd88706fa2e3aa83f2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:33.632239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:33.633877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:33 2022] GET /identity/v3/users/0fa271b4851840cd88706fa2e3aa83f2 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:33.640539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:33.753038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61a5c-cd2b-4e08-81a6-acac70d23f4e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:33.782813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:33.791417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:33.792002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:33.792293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:33.792557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:33.863766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:33.866000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:33.981326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:33.988317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:34.088868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:34.100769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc983a-debd-4064-b834-58cb9380b1b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:34.138409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:34.139131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:34.139404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:34.139666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:34.140844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:34.142277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d77b4-9c11-4e26-9b77-005f71cdd33a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:34.143885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:33 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:34.150708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:34.178926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:34.255679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67300-a5c6-42fa-8223-d188fee0586e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:34.291167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:34.291749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:34.292020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:34.292310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:34.293023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=18a806e76e2949d6a72dc62f9875785c, user.id=0fa271b4851840cd88706fa2e3aa83f2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:34.293321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:34.294736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7424082-3af3-4277-80ef-fc967adc6453 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:34.300063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:34 2022] GET /identity/v3/role_assignments?user.id=0fa271b4851840cd88706fa2e3aa83f2&scope.project.id=18a806e76e2949d6a72dc62f9875785c&role.id=7f08073ba42c48ff8060c007983602f6 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:36.249272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0370f-4fec-4d8b-8927-0c31cb1ea99f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:36.249812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0370f-4fec-4d8b-8927-0c31cb1ea99f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:36.249812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0370f-4fec-4d8b-8927-0c31cb1ea99f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:36.251054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1118] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:36.257280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:36.257951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:36.258108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:36.285705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:36.286919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:36.357771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:36.380926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0370f-4fec-4d8b-8927-0c31cb1ea99f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:36.486842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0370f-4fec-4d8b-8927-0c31cb1ea99f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:36.522468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:36.523085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:36.523365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:36.523631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] PATH_INFO: `/v3/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:36.525175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] RBAC: Authorizing `identity:get_role(role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:36.526582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:36.527911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:36 2022] GET /identity/v3/roles/7f08073ba42c48ff8060c007983602f6 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:36.534312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:36.558867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:36.647821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1dc46-834c-4ec8-81ee-144e7790ac06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:36.684139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:36.684779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:36.685032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:36.685309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] PATH_INFO: `/v3/users/0fa271b4851840cd88706fa2e3aa83f2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:36.686470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] RBAC: Authorizing `identity:get_user(user_id=0fa271b4851840cd88706fa2e3aa83f2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:36.687912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:36.689512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:36 2022] GET /identity/v3/users/0fa271b4851840cd88706fa2e3aa83f2 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:36.698403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:36.727628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:36.824376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f808273-268d-40e4-afd8-75cb85909163 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:36.859982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:36.860612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:36.860878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:36.861138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:36.862361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:36.863769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7a254-c071-4ae5-8504-aaf56dd52150 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:36.865379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:36 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:36.875376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:36.986338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:36.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225fc1-1ea9-44b4-b96c-7246fa0929c2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:37.033343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:37.033982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:37.034259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:37.034527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c/users/0fa271b4851840cd88706fa2e3aa83f2/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:37.037093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=18a806e76e2949d6a72dc62f9875785c, user_id=0fa271b4851840cd88706fa2e3aa83f2, role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:37.038520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a565f-569c-483b-97ee-856f6ef12ba1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:37.058158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:06:36 2022] PUT /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c/users/0fa271b4851840cd88706fa2e3aa83f2/roles/7f08073ba42c48ff8060c007983602f6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:06:39.005752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003438-882d-4ab9-9307-d0abead1b57b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.006569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003438-882d-4ab9-9307-d0abead1b57b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.006680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003438-882d-4ab9-9307-d0abead1b57b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.008819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1124] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:39.015056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.015710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.015809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.046533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:39.047705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:39.118672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:39.149830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003438-882d-4ab9-9307-d0abead1b57b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:39.256187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:39.264981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003438-882d-4ab9-9307-d0abead1b57b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:39.301470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:39.302073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.302343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.302604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] PATH_INFO: `/v3/roles/7f08073ba42c48ff8060c007983602f6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.304099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] RBAC: Authorizing `identity:get_role(role_id=7f08073ba42c48ff8060c007983602f6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:39.305511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:39.306810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:39 2022] GET /identity/v3/roles/7f08073ba42c48ff8060c007983602f6 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:39.313440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:39.423320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:39.449498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1385c-bb4d-43ed-8fd2-5ff88a5c1267 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:39.486930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:39.487545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.487815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.488085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] PATH_INFO: `/v3/users/0fa271b4851840cd88706fa2e3aa83f2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.489361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] RBAC: Authorizing `identity:get_user(user_id=0fa271b4851840cd88706fa2e3aa83f2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:39.490793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:39.492377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:39 2022] GET /identity/v3/users/0fa271b4851840cd88706fa2e3aa83f2 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:39.498640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:39.602850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0f19e-fa29-4fb1-a909-fea08e8f3a67 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:39.642846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:39.643452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.643748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.644010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.679561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:39.718272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:39.719722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:39.821635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:39.828032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:39.935395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e9412-27e1-4997-be85-445f55643330 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:39.975690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:39.976312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:39.976595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:39.976857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] PATH_INFO: `/v3/projects/18a806e76e2949d6a72dc62f9875785c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:39.978086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] RBAC: Authorizing `identity:get_project(project_id=18a806e76e2949d6a72dc62f9875785c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:39.979504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6784e1-cbeb-447b-9546-1391eb2841a5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:39.981171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:39 2022] GET /identity/v3/projects/18a806e76e2949d6a72dc62f9875785c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:39.988201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:40.055256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:40.097282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0eada-ddac-4d66-9fa8-379a6d3928c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:40.133925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:40.134780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:40.135127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:40.135480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:40.136641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] RBAC: Adding query filter params (role.id=7f08073ba42c48ff8060c007983602f6, scope.project.id=18a806e76e2949d6a72dc62f9875785c, user.id=0fa271b4851840cd88706fa2e3aa83f2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:40.137057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:40.139375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca234cc-12e8-4ecb-975e-d3b8d6953430 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:40.149749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:39 2022] GET /identity/v3/role_assignments?user.id=0fa271b4851840cd88706fa2e3aa83f2&scope.project.id=18a806e76e2949d6a72dc62f9875785c&role.id=7f08073ba42c48ff8060c007983602f6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:42.159582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775895f-70d8-4758-b26d-0b9d05656f23 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:42.160504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775895f-70d8-4758-b26d-0b9d05656f23 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:42.160504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775895f-70d8-4758-b26d-0b9d05656f23 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:42.162580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1131] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:42.168011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:42.168449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:42.168556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:42.188304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:42.189488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:42.262409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:42.285853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775895f-70d8-4758-b26d-0b9d05656f23 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:42.304621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:42.411312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775895f-70d8-4758-b26d-0b9d05656f23 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:42.448304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:42.448852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:42.449118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:42.449384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:42.450533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:42.451971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d6332-5ea9-40c1-a12a-eec2bca9fa7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:42.453540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:42.462986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:42.526992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:42.569721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af3a5-9502-4b3e-8ff8-550317d1d2ed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:42.610074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:42.611068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:42.611556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:42.612021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:42.612947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:42.614679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77467679-e06d-4460-ba4b-30e1d6604a91 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:42.647715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:06:42 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:44.681712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ce728-b0d9-46ad-9645-57a7c775ae35 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:44.682640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ce728-b0d9-46ad-9645-57a7c775ae35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:44.682640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ce728-b0d9-46ad-9645-57a7c775ae35 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:44.684725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1135] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:44.691062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:44.691822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:44.691932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:44.720372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:44.721557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:44.801275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:44.829774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ce728-b0d9-46ad-9645-57a7c775ae35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:44.923574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:44.942229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ce728-b0d9-46ad-9645-57a7c775ae35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:44.980225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:44.980874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:44.981142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:44.981415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:44.982622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:44.984043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65eb59-ed38-4c64-b46a-5945a4707115 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:44.985612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:06:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:44.994881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:45.044727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:45.105995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec210f3-d468-4618-8106-9cf0baefece1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:45.141845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:45.142775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:45.143199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:45.143624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:45.144676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:45.147305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041755a-27ac-42fa-a502-9d41b3fa6c73 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:45.178564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:06:44 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:06:47.203755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0564-1d4c-4b53-9cd7-85fc48783b1e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.204614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0564-1d4c-4b53-9cd7-85fc48783b1e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.204614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab0564-1d4c-4b53-9cd7-85fc48783b1e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:47.206635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1139] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:47.212872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.213320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.213410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:47.234411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:47.235594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:47.311121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:47.334092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab0564-1d4c-4b53-9cd7-85fc48783b1e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:47.453285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab0564-1d4c-4b53-9cd7-85fc48783b1e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:47.488199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:47.488777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.489051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.489322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:47.500283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:47.504889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:47.505951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:47.507065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:47.514585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:47.543335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:47.619624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d418-95a4-4784-af1f-0f6e3a709e2b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:47.655108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:47.655698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.655967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.656231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:47.656920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:47.657219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:47.658582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:47.666350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:47.673089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:47.698834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:47.794267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cde99b-a4d6-42ce-a819-5c3fdc10318d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:47.832656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:47.833286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.833534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.833796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] PATH_INFO: `/v3/users/50ace14135fd44778ed127da56cb7980` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:47.846978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] RBAC: Authorizing `identity:get_user(user_id=50ace14135fd44778ed127da56cb7980)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:47.848474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:47.850039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:47 2022] GET /identity/v3/users/50ace14135fd44778ed127da56cb7980 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:47.856334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:47.948896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:47.962348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8af9eb-11bc-4e44-a7dc-7a01d0353e27 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:47.996521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:47.997116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:47.997385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:47.997649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:48.069569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:48.070937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:48.181470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:48.195689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:48.202245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:48.272697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:48.304182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e441d-cdef-4fc8-b3c8-85c697ef97db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:48.338866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:48.339412 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:48.339680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:48.339950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] PATH_INFO: `/v3/projects/29cfd68790994c4f82728c11e159543f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:48.341097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] RBAC: Authorizing `identity:get_project(project_id=29cfd68790994c4f82728c11e159543f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:48.342534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a8631-88ed-40e1-bf26-0966c0f950d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:48.344120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:48 2022] GET /identity/v3/projects/29cfd68790994c4f82728c11e159543f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:48.350630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:48.465833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6f59c-35aa-4ba0-bf26-608299ebd595 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:48.506100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:48.506702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:48.506968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:48.507227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:48.507949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=29cfd68790994c4f82728c11e159543f, user.id=50ace14135fd44778ed127da56cb7980) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:48.508248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:48.509725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a3a9d-d559-4a96-9333-a89ee5a5319a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:48.515246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:48 2022] GET /identity/v3/role_assignments?user.id=50ace14135fd44778ed127da56cb7980&scope.project.id=29cfd68790994c4f82728c11e159543f&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:50.473581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6bce6-2cf9-423d-a1be-35b3482c4471 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:50.474157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6bce6-2cf9-423d-a1be-35b3482c4471 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:50.474157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6bce6-2cf9-423d-a1be-35b3482c4471 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:50.475443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1147] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:50.481940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:50.482648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:50.483082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:50.506968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:50.508187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:50.561064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:50.583340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:50.616628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6bce6-2cf9-423d-a1be-35b3482c4471 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:50.730025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6bce6-2cf9-423d-a1be-35b3482c4471 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:50.766943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:50.767555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:50.767824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:50.768088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:50.775975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:50.780372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:50.785288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:50.786343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:50.792488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:50.793354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:50.901303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee16d1c-04dd-417a-8725-e255f3cf5c54 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:50.936465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956380-171a-440c-b43d-627cec625d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:50.937059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956380-171a-440c-b43d-627cec625d26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:50.937298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956380-171a-440c-b43d-627cec625d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:50.937592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956380-171a-440c-b43d-627cec625d26 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:50.938234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30956380-171a-440c-b43d-627cec625d26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:50.938531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30956380-171a-440c-b43d-627cec625d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:50.939883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30956380-171a-440c-b43d-627cec625d26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:50.947885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:50.954589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:51.057920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005e19-8242-409e-b306-d8fdbbda63b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:51.097426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:51.098018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:51.098289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:51.098556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] PATH_INFO: `/v3/users/50ace14135fd44778ed127da56cb7980` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:51.099766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] RBAC: Authorizing `identity:get_user(user_id=50ace14135fd44778ed127da56cb7980)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:51.101194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:51.102711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:50 2022] GET /identity/v3/users/50ace14135fd44778ed127da56cb7980 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:51.112174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956380-171a-440c-b43d-627cec625d26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:51.166454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:51.228564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956380-171a-440c-b43d-627cec625d26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:51.267429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:51.268037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:51.268347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:51.268599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] PATH_INFO: `/v3/projects/29cfd68790994c4f82728c11e159543f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:51.269720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] RBAC: Authorizing `identity:get_project(project_id=29cfd68790994c4f82728c11e159543f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:51.271466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e70fd3-46b4-4b5f-b0cd-7120a13b02a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:51.273110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:51 2022] GET /identity/v3/projects/29cfd68790994c4f82728c11e159543f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:51.282692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:51.353827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:51.388677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c0f36-64ca-442c-b195-9303e80ffcf3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:51.421813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:51.422450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:51.422725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:51.422995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] PATH_INFO: `/v3/projects/29cfd68790994c4f82728c11e159543f/users/50ace14135fd44778ed127da56cb7980/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:51.425129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] RBAC: Authorizing `identity:create_grant(project_id=29cfd68790994c4f82728c11e159543f, user_id=50ace14135fd44778ed127da56cb7980, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:51.426545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7bbc4-e6bf-4258-b4e7-bd94e31e9a52 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:51.446768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:06:51 2022] PUT /identity/v3/projects/29cfd68790994c4f82728c11e159543f/users/50ace14135fd44778ed127da56cb7980/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:06:53.406373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87de18-c256-42d6-b2bd-1c1394e8fb7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:53.406957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87de18-c256-42d6-b2bd-1c1394e8fb7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:53.406957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87de18-c256-42d6-b2bd-1c1394e8fb7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:53.408259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1154] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:53.414512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:53.415226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:53.415393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:53.442849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:53.444016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:53.516233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:53.539803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87de18-c256-42d6-b2bd-1c1394e8fb7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:53.654512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87de18-c256-42d6-b2bd-1c1394e8fb7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:53.667576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:53.702291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:53.702904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:53.703168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:53.703484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:53.714770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:06:53.719806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:53.720911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:06:53.721931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:53.727441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:53.785552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:53.838364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b5066-8de3-478d-83a2-932d008cbcde None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:53.875144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:53.875727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:53.875998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:53.876295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:53.876936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:53.877231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:53.878569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:53.887150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:06:53.894424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:54.010908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118ce77-5ff7-4150-94cc-a8866a5ba135 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:54.047034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:54.047639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:54.047908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:54.048168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] PATH_INFO: `/v3/users/50ace14135fd44778ed127da56cb7980` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:54.049408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] RBAC: Authorizing `identity:get_user(user_id=50ace14135fd44778ed127da56cb7980)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:54.050825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:54.052371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:06:53 2022] GET /identity/v3/users/50ace14135fd44778ed127da56cb7980 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:54.058460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:54.164826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997c08-ca3c-4a35-8922-16b86d611ccc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:54.182712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:54.212674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:54.213449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:54.213763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:54.214073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:54.283358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:54.284739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:54.390298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:54.397109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:54.442645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:54.518147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17febd91-bcc1-40ef-94a8-8d10dfe7728b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:54.554779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:54.555327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:54.555597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:54.555862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] PATH_INFO: `/v3/projects/29cfd68790994c4f82728c11e159543f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:54.557028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] RBAC: Authorizing `identity:get_project(project_id=29cfd68790994c4f82728c11e159543f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:54.558458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0dff1-5507-4b5b-9a06-e8ab798ef83b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:54.560052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:06:54 2022] GET /identity/v3/projects/29cfd68790994c4f82728c11e159543f => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:54.567257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:54.590070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:54.671259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26380de-8d0a-4e4e-85a5-04d18a0c9739 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:54.709657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:54.710237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:54.710503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:54.710768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:54.711509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=29cfd68790994c4f82728c11e159543f, user.id=50ace14135fd44778ed127da56cb7980) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:54.711810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:54.713247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228bce3-d4db-4811-9110-b6b6409905c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:54.719143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:06:54 2022] GET /identity/v3/role_assignments?user.id=50ace14135fd44778ed127da56cb7980&scope.project.id=29cfd68790994c4f82728c11e159543f&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:56.698302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf68b4-435e-45c6-a558-53775ca27ae3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:56.698862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf68b4-435e-45c6-a558-53775ca27ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:56.698862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf68b4-435e-45c6-a558-53775ca27ae3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:56.700157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1162] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:56.706658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:56.707345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:56.707521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:56.736643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:56.737837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:56.809134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:56.837762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf68b4-435e-45c6-a558-53775ca27ae3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:56.961632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf68b4-435e-45c6-a558-53775ca27ae3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:56.971678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:56.999903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:57.000516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:57.000794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:57.001057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:57.074888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:57.076392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:57.178995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:06:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:06:57.185865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:57.200416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:57.304907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055e6f8-2be0-43fb-ab50-fc61dfe41d9c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:57.342913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:57.343523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:57.343792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:57.344055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:57.350904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:57.352361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c3c5d91d-8e73-4dad-bc13-e54c29730a0c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:06:57.363582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:06:57.364796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:06:57.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:06:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:06:57.371329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:57.386643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:57.475507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f058a9-b9cc-412c-9d83-1732c014855e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:57.511185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:57.511774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:57.512046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:57.512305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:57.512959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:06:57.513281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:57.514655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4dee0-ca94-4179-954c-43654dc4bc92 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:57.522156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 9 11:06:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:06:59.509180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293fdbf-e99e-4c87-aca7-3a5d38f09bb7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:59.510123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293fdbf-e99e-4c87-aca7-3a5d38f09bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:59.510123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293fdbf-e99e-4c87-aca7-3a5d38f09bb7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:59.512166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1167] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:06:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:06:59.517636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b63063-4b06-445c-8b5b-a89177fc252d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:59.518047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b63063-4b06-445c-8b5b-a89177fc252d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:59.518133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b63063-4b06-445c-8b5b-a89177fc252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:59.537868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-56b63063-4b06-445c-8b5b-a89177fc252d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:06:59.539066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56b63063-4b06-445c-8b5b-a89177fc252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:06:59.608287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:06:59.631849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293fdbf-e99e-4c87-aca7-3a5d38f09bb7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:06:59.714610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:06:59.753584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293fdbf-e99e-4c87-aca7-3a5d38f09bb7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:06:59.789582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:06:59.790101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:06:59.790371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:06:59.790630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:06:59.791280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:06:59.792666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d10c36-e833-4f64-91c2-0baa321202cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:06:59.812818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Tue Aug 9 11:06:59 2022] POST /identity/v3/domains => generated 267 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:07:01.823338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442c98-1f18-43e8-aabf-0b3d812636aa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:01.823946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442c98-1f18-43e8-aabf-0b3d812636aa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:01.823946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442c98-1f18-43e8-aabf-0b3d812636aa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:01.825232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1170] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:07:01.831608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:01.832386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:01.832575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:01.855031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:07:01.856199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:07:01.924924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:07:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:07:01.948336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7442c98-1f18-43e8-aabf-0b3d812636aa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:01.952849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:02.061405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7442c98-1f18-43e8-aabf-0b3d812636aa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:02.109675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:02.110480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:02.110853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:02.111201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] PATH_INFO: `/v3/domains/bca5f4ccd148485a83be2e070c069433` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:02.126831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bca5f4ccd148485a83be2e070c069433)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:02.128313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60cf10-d614-4915-83b2-b23a9889beb3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:02.129874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Aug 9 11:07:01 2022] GET /identity/v3/domains/bca5f4ccd148485a83be2e070c069433 => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:02.135966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:02.199302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:02.242605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbd35a-166d-41b5-a906-8cee368f748a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:02.277757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:02.278341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:02.278616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:02.279407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:02.280047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:02.281538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b7929-018f-46b6-9513-7fa4a33663e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:02.314075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:07:02 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:07:04.335858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1867d1-0061-4a36-967a-70e8451230b5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:04.336809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1867d1-0061-4a36-967a-70e8451230b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:04.336809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1867d1-0061-4a36-967a-70e8451230b5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:04.338917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1174] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:07:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:07:04.347753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:04.348187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:04.348323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:04.375322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:07:04.376530 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:07:04.446668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:07:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:07:04.472803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1867d1-0061-4a36-967a-70e8451230b5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:04.539413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:04.581007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1867d1-0061-4a36-967a-70e8451230b5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:04.615510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:04.616052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:04.616349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:04.616615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] PATH_INFO: `/v3/domains/bca5f4ccd148485a83be2e070c069433` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:04.617758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bca5f4ccd148485a83be2e070c069433)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:04.619195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2717c-1546-4c4e-94fd-b588ab98b039 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:04.620769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Aug 9 11:07:04 2022] GET /identity/v3/domains/bca5f4ccd148485a83be2e070c069433 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:04.627322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:04.681134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:04.739409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe28ce-8f1a-4620-814b-ddf5ec3fb624 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:04.774581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:04.775166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:04.775439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:04.775705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:04.776334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:04.777788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639fae8-e2bd-4bcd-9899-15a5fe82c272 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:04.802212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:07:04 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:07:06.803660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a12b-cd33-4456-8d2b-02dc6b91da26 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:06.804261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a12b-cd33-4456-8d2b-02dc6b91da26 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:06.804261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1a12b-cd33-4456-8d2b-02dc6b91da26 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:06.805563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1178] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:07:06.812612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:06.813372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:06.813501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:06.843510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:07:06.844737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:07:06.922041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:07:06.945896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1a12b-cd33-4456-8d2b-02dc6b91da26 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:07.062587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1a12b-cd33-4456-8d2b-02dc6b91da26 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:07.091032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:07.098008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:07.098618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:07.098885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:07.099146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:07.110996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:07:07.115559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:07:07.122274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:07:07.123304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:07:07.125635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:07.162685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:07.241978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce0d86-b81a-4ac8-a3d4-ecea3c79e105 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:07.277083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:07.277678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:07.277950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:07.278215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:07.279027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:07:07.279336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:07.280704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:07.288913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:07.296552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:07.411234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011080a5-be6f-4174-ab92-ebd817417dec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:07.446938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:07.447547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:07.447818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:07.448085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] PATH_INFO: `/v3/users/1f5cbabbd3f74368af9608313ae23bd3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:07.463051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] RBAC: Authorizing `identity:get_user(user_id=1f5cbabbd3f74368af9608313ae23bd3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:07.464546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:07.466578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:07:07 2022] GET /identity/v3/users/1f5cbabbd3f74368af9608313ae23bd3 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:07.472759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:07.578227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:07.592517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05abec-22d0-432a-87d1-b611385b5064 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:07.627681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:07.628257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:07.628549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:07.628811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:07.697080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:07.698441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:07.797238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:07.807173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:07:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:07:07.813549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:07.871277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:07.933174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08fac5-4530-4ae5-a4ab-1f2f8823a90a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:07.968442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:07.968987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:07.969252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:07.969513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] PATH_INFO: `/v3/projects/29c9ac603f23458f84c07ed1916a16b2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:07.970660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=29c9ac603f23458f84c07ed1916a16b2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:07.972082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4aaf5-ff0a-4e5b-a4dc-38f150fee9b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:07.973709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:07:07 2022] GET /identity/v3/projects/29c9ac603f23458f84c07ed1916a16b2 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:07:07.982928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:08.098614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163a70e-89b0-4c3f-92e3-7d244e503489 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:08.134199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:08.134775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:08.135044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:08.135307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:08.136008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=29c9ac603f23458f84c07ed1916a16b2, user.id=1f5cbabbd3f74368af9608313ae23bd3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:07:08.136305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:08.137744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad29f7f-eb80-4ae5-b496-d4f5b011605a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:08.143239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:07:07 2022] GET /identity/v3/role_assignments?user.id=1f5cbabbd3f74368af9608313ae23bd3&scope.project.id=29c9ac603f23458f84c07ed1916a16b2&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:10.095312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc64a-25ba-4b52-9f8b-d9b32ba9cea2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.095897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc64a-25ba-4b52-9f8b-d9b32ba9cea2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.095897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cc64a-25ba-4b52-9f8b-d9b32ba9cea2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.097205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1186] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:07:10.103431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.104186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.104363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.133525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:07:10.135363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:07:10.189361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:10.208801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:07:10.231151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cc64a-25ba-4b52-9f8b-d9b32ba9cea2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:10.355432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cc64a-25ba-4b52-9f8b-d9b32ba9cea2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:10.371885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:10.392815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:10.393423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.393694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.393950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.404749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:07:10.409262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:07:10.410335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:10.411315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:07:10.412444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:07:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:07:10.416432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:10.522928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5adf2e-e1c3-488c-a917-9194cd7b26b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:10.559112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:10.559756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.560006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.560288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.560988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:07:10.561280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:10.562648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:10.572195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:07:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:10.579052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:10.709097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b2ed0-7ea7-4122-84e5-b75681d032c6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:10.748794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:10.749407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.749673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.749932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] PATH_INFO: `/v3/users/1f5cbabbd3f74368af9608313ae23bd3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.751102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] RBAC: Authorizing `identity:get_user(user_id=1f5cbabbd3f74368af9608313ae23bd3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:10.752538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:10.754090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:07:10 2022] GET /identity/v3/users/1f5cbabbd3f74368af9608313ae23bd3 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:10.760050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:10.810680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:10.862731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabe875-9f31-46df-a67d-5cac6dcec634 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:10.902740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:10.903527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:10.903661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:10.903920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] PATH_INFO: `/v3/projects/29c9ac603f23458f84c07ed1916a16b2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:10.905168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] RBAC: Authorizing `identity:get_project(project_id=29c9ac603f23458f84c07ed1916a16b2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:10.906574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b57a0c7-49c6-41fd-b1e8-a185f9f172a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:10.908173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:07:10 2022] GET /identity/v3/projects/29c9ac603f23458f84c07ed1916a16b2 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:10.918059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:10.971247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:11.042340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc4a2-d356-477c-84aa-2d6d5a477349 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:11.082062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:11.082626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:11.082894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:11.083158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] PATH_INFO: `/v3/projects/29c9ac603f23458f84c07ed1916a16b2/users/1f5cbabbd3f74368af9608313ae23bd3/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:11.085248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] RBAC: Authorizing `identity:create_grant(project_id=29c9ac603f23458f84c07ed1916a16b2, user_id=1f5cbabbd3f74368af9608313ae23bd3, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:11.086687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e80ab-33db-4dd4-9716-8bfd12e1ead5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:11.101332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:07:10 2022] PUT /identity/v3/projects/29c9ac603f23458f84c07ed1916a16b2/users/1f5cbabbd3f74368af9608313ae23bd3/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:07:13.048883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d8ee-6eec-44f4-b080-71a3d63199a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.049420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d8ee-6eec-44f4-b080-71a3d63199a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.049420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d8ee-6eec-44f4-b080-71a3d63199a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.051436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1193] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:07:13.059551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.061050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.061516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:07:13.087275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:07:13.163586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:07:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:07:13.203230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d8ee-6eec-44f4-b080-71a3d63199a3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:13.332865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d8ee-6eec-44f4-b080-71a3d63199a3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:13.349101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:13.372653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:13.373419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.373694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.373961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.386140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:07:13.391179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:07:13.392243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:07:13.393159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:07:13.399561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:13.444833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:13.525452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ff1e0-1659-412d-a71d-7fb4988dd253 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:13.560996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:13.561582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.561848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.562107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.562758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:07:13.563048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:13.564447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:13.572418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:13.579368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:13.691119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f78ae-4ccc-47f5-8b7d-f0039e4edcd4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:13.725884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:13.726423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.726691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.726959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] PATH_INFO: `/v3/users/1f5cbabbd3f74368af9608313ae23bd3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.728100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] RBAC: Authorizing `identity:get_user(user_id=1f5cbabbd3f74368af9608313ae23bd3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:13.729541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:13.731083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 9 11:07:13 2022] GET /identity/v3/users/1f5cbabbd3f74368af9608313ae23bd3 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:13.737408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:13.832634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:13.839791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450966-7269-4a9a-98eb-e375ab88c6d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:13.876158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:13.876772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:13.877042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:13.877299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:13.954350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:13.955798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:14.067487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:14.076920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:07:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:07:14.083313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:14.131893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:14.199542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8c8a8-d8d8-4a0c-935a-fb784e5945e5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:14.233553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:14.234160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:14.234426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:14.234687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] PATH_INFO: `/v3/projects/29c9ac603f23458f84c07ed1916a16b2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:14.235807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] RBAC: Authorizing `identity:get_project(project_id=29c9ac603f23458f84c07ed1916a16b2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:14.237256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484caa54-b14b-470b-9bce-dba8079f14bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:14.238880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 9 11:07:14 2022] GET /identity/v3/projects/29c9ac603f23458f84c07ed1916a16b2 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:14.248523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:07:14.356521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87546a9-9971-425f-bc79-4771237674a4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:07:14.391055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:07:14.391642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:07:14.391904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:07:14.392174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:07:14.392929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=29c9ac603f23458f84c07ed1916a16b2, user.id=1f5cbabbd3f74368af9608313ae23bd3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:07:14.393228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:07:14.394641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba2ad2-b397-456a-a3e7-269377336e55 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:07:14.400795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:07:14 2022] GET /identity/v3/role_assignments?user.id=1f5cbabbd3f74368af9608313ae23bd3&scope.project.id=29c9ac603f23458f84c07ed1916a16b2&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:07:24.228984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=44 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:07:24.398134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=70 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:09.267852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fce6e-6283-41e9-88a8-bca94564bb1d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:09.268489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fce6e-6283-41e9-88a8-bca94564bb1d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:09.268680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fce6e-6283-41e9-88a8-bca94564bb1d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:09.270830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1201] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:09.279790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:09.279790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:09.279790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:09.296210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:09.297520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:09.379916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:09.405435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fce6e-6283-41e9-88a8-bca94564bb1d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:09.539798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fce6e-6283-41e9-88a8-bca94564bb1d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:09.575684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:09.576448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:09.576723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:09.576992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:09.588631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:09.590812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:09.606147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:09.607268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:09.608285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:09.608285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:09.608285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:09.608285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:09.616732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:09.733103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291a3d4-f54b-4c97-ac54-873d36e63321 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:09.778756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:09.787748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:09.788395 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:09.788671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:09.788979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:09.789893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:09.790186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:09.791777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d6222-ba13-416a-9b19-020bb141b745 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:09.808925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:09.814162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:09.881435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:09.946962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd409a53-7ffb-432d-9b1e-5a2e903df9c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:09.998718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:09.999429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:09.999745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:10.000043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:10.000921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:10.002698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb782c7-9feb-450e-9a77-1518e449906f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:10.028681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 9 11:08:09 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:08:12.308156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3294-9f2c-4a6e-bb1a-99e2d31b5682 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:12.308849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3294-9f2c-4a6e-bb1a-99e2d31b5682 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:12.308959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3294-9f2c-4a6e-bb1a-99e2d31b5682 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:12.310294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1206] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:12.316414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:12.316952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:12.317043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:12.338668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:12.339944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:12.418567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:12.444504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3294-9f2c-4a6e-bb1a-99e2d31b5682 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:12.561011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3294-9f2c-4a6e-bb1a-99e2d31b5682 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:12.596729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:12.597375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:12.597669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:12.597894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:12.610447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:12.615198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:12.616467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:12.617322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:08:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:12.622872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:12.676022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:12.750760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff470b-480e-4f9e-aa9d-3a0d81059d65 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:12.789750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:12.790650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:12.790919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:12.791185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:12.791837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:12.792222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:12.793671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:12.802076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:08:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:12.807944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:12.866386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:12.924568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad0e56-2005-4a2d-88d4-4a3d252e584d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:12.963786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:12.964940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:12.965455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:12.965691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:12.976650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:12.978081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:12.992737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:12.994356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:12.995711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:12.995711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:12.995711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:12.998560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:13.000217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:13.108583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:13.111539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff4719-6220-449e-92e4-2e01e0605f8a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:13.150214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:13.150783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:13.151174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:13.151444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:13.152184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:13.152566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:13.154126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:13.178673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:13.180725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:13.302165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37926c-2164-4716-a83c-34a84a07dbbf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:13.304160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:13.341256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:13.342115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:13.342208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:13.343985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:13.356812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:13.358291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:13.370855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:13.372304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:13.373657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:13.374715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:13.378414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:13.487582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf62ac-1db4-42a9-935c-293dc2fc7060 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:13.528157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:13.529034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:13.529494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:13.529883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:13.531067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:13.531554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:13.536140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:13.561016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:13.567272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:13.701636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff097211-c95c-4e9a-8b4d-ae92c4b58ef5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:13.747359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:13.747926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:13.748204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:13.748510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:13.786004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:13.829958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:13.831545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:13.944337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:08:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:08:13.950270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:13.971170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:14.063965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a6389-c2eb-4b8c-ad16-df95d616a28e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:14.100779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:14.101416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:14.101830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:14.101983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:14.113153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:14.114591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:14.130608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:14.132036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:14.133160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:14.133160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:14.133160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:14.136902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:14.137829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:14.204964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:14.247568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d81c-3b22-4ca6-bb27-2ed777d95d9e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:14.291539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:14.292365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:14.292644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:14.293197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:14.293972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:14.294438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:14.295970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:14.309076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:14.314897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:14.397216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:14.435275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb232c-f483-44ac-8f67-6d73f97c77b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:14.475532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:14.476191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:14.476528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:14.476795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:14.484591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:14.486095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:14.499203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:14.505159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:14.506404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:08:14.507178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:14.635480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b69f-fedf-4dd0-9c0c-cfc4be3ae101 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:14.663794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:14.679269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:14.680197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:14.680528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:14.680957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:14.681702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:14.682071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:14.684069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd9e9-952a-4c03-9d8e-76a361da5a09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:14.703686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:14.705024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:14.823985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5128b8-2afb-4ee8-a3cb-5a7f7cccc558 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:14.856086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:14.864042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:14.864832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:14.865186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:14.865583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:14.866420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=9792491cfe3c489a83639e77a27bde04) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:14.866862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:14.868526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfe5ae-216e-4d19-9242-00e4befeacb6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:14.874840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:08:14 2022] GET /identity/v3/role_assignments?user.id=9792491cfe3c489a83639e77a27bde04&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:16.867049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49221a-d249-4713-a94a-2c3ffb3c33ba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:16.867701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49221a-d249-4713-a94a-2c3ffb3c33ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:16.867701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49221a-d249-4713-a94a-2c3ffb3c33ba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:16.868937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1220] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:16.875274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97823444-7cc4-4989-b313-b4d393935727 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:16.875718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97823444-7cc4-4989-b313-b4d393935727 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:16.875974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97823444-7cc4-4989-b313-b4d393935727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:16.898121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-97823444-7cc4-4989-b313-b4d393935727 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:16.899314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97823444-7cc4-4989-b313-b4d393935727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:16.978764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:17.003996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49221a-d249-4713-a94a-2c3ffb3c33ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.119907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49221a-d249-4713-a94a-2c3ffb3c33ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:17.161961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:17.162670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:17.162957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:17.163222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:17.177162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:17.182329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:17.183500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:17.184772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:17.190123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97823444-7cc4-4989-b313-b4d393935727 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.306867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97823444-7cc4-4989-b313-b4d393935727 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:17.331150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:17.342626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:17.343301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:17.343569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:17.343858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:17.344618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:17.344944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:17.346388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:17.355906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:17.361916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.411015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:17.471568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9015d54b-abf3-41db-886f-0f791240993c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:17.505848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:17.506476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:17.506766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:17.507015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:17.518003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:17.519794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:17.534668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:17.535685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:17.537364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:17.537364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:17.537364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:17.537364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:17.541750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.654087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db5880-7d52-4eea-a399-c78fb601ff77 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:17.696163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:17.696734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:17.697007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:17.697274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:17.697913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:17.698427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:17.700191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:17.712024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:17.718538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.816509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:17.825039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f70d7-c335-49cf-91ba-f2fee3e312ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:17.861140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:17.861771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:17.862046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:17.862322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:17.871062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:17.872599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:17.887782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:17.888955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:17.889992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:17.895285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:17.943218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:17.996260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a28059-ea49-46d7-90de-d4de24579390 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.035730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.036375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.036685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.036999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.037741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:18.038101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.039605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.058132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:18.064011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:18.171886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1d6e9-cabb-485e-bcfe-c42966c33d50 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.206137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.206754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.207028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.207290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.220407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.221861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:18.236466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:18.237763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:18.238741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:18.238741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:18.238741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:18.238741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:08:18.243721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:18.357639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd82e0-e8da-4606-b6ad-5eae9a8b32b9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.359894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:18.393655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.394256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.394840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.395106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.395741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:18.396017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.397408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.408511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:18.414294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:18.424641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:18.534292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c2bc-1e88-4655-a99d-97a20df890d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.572875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.573583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.573871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.574144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.583198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.584747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:18.598399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:18.599668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:18.605837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:18.606360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:18.733323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c52be8-1691-472b-9836-af953cb9ce2f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.775299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.776026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.776836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.777197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.778392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:18.779006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.780893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6cc41-9cf8-4fcd-99c2-54d7ad63512f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.794242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:18.800813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:18.873170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:18.915531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f264-6389-4c95-9a6c-77d5109ad793 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:18.953812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:18.954470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:18.954763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:18.955014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/9792491cfe3c489a83639e77a27bde04/roles/e8b8ffe71e224b9788793ca5938d2513` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:18.970493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=9792491cfe3c489a83639e77a27bde04, role_id=e8b8ffe71e224b9788793ca5938d2513)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:18.972286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c2db-eaef-488d-b33e-20e5a45963c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:18.986010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:08:18 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/9792491cfe3c489a83639e77a27bde04/roles/e8b8ffe71e224b9788793ca5938d2513 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:08:21.290820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad882-bc0b-4799-8b26-f21afc737563 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:21.291381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad882-bc0b-4799-8b26-f21afc737563 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:21.291585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ad882-bc0b-4799-8b26-f21afc737563 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:21.293508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1233] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:21.299476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:21.299971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:21.300162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:21.320983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:21.322183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:21.404950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:21.429203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ad882-bc0b-4799-8b26-f21afc737563 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:21.460500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:21.548025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ad882-bc0b-4799-8b26-f21afc737563 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:21.584257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:21.584898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:21.585168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:21.585431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:21.595831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:21.600507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:21.601744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:21.602590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 9 11:08:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:21.608195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:21.658385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:21.732522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876368-1db8-463e-958c-dc6014c93b3a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:21.770931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:21.771535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:21.771802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:21.772057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:21.772747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:21.773039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:21.774405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:21.783601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 9 11:08:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:21.789801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:21.877893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:21.897617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f7580-70b0-48d0-8748-b939c32d0b46 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:21.938255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:21.938970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:21.939247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:21.939518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:21.950982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:21.952813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:21.967770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:21.969053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:21.969960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:21.969960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:21.969960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:21.969960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:21.974880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:22.078156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177fe9-6d90-465b-afe0-3118ab1887c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:22.087247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:22.115413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:22.116180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:22.116468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:22.116761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:22.117431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:22.117715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:22.119151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:22.131251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:22.137569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:22.247926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6d2b-ca5c-40a9-a49b-61780198b684 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:22.286016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:22.286643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:22.286919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:22.287187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:22.290223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:22.297562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:22.299153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:22.311709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:22.313030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:22.314430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:22.319010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:22.427231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152ad03-f724-41b1-a85e-7ee2ff4643df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:22.464861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:22.466576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:22.467256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:22.467717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:22.469316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:22.469846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:22.472676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:22.491572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:22.497919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:22.606553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b91121-cb61-4f22-8ab2-bbf1e3d4d6cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:22.640218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:22.640831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:22.641109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:22.641372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:22.712603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:22.720040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:22.721935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:22.835177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:08:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:08:22.841311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:22.875006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:22.969358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b8326-a684-415d-a71f-9fda366fcbee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:23.013707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:23.014338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:23.014614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:23.014876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:23.027467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:23.029232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:23.045953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:23.047235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:23.048114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:23.048114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:23.048114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:23.048114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:23.053060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:23.167531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:23.174593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a75def-3e8e-4c35-8d42-a723b291a347 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:23.217547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:23.218157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:23.218430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:23.218696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:23.219335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:23.219684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:23.221346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:23.234806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:23.241374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:23.331585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:23.357259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299448-29a1-4eb9-b967-f3e0478d6536 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:23.400381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:23.401030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:23.401285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:23.401600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:23.410852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:23.412681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:23.427068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:23.428200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:23.429137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:23.436036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:23.548405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b7118-af48-44d0-ab20-cad0b8985abe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:23.585685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:23.586389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:23.586748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:23.587051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:23.587843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:23.588187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:23.589978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b7b71-a593-42ff-bd8f-686a1cebe365 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:23.603525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:23.610360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:23.736499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39b8a-046d-4b07-818b-6895a5eba8d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:23.776647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:23.777253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:23.777526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:23.777788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:23.778509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] RBAC: Adding query filter params (role.id=e8b8ffe71e224b9788793ca5938d2513, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=9792491cfe3c489a83639e77a27bde04) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:23.778843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:23.780257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356397ab-3202-47ca-93f3-2b83767c07dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:23.797295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:08:23 2022] GET /identity/v3/role_assignments?user.id=9792491cfe3c489a83639e77a27bde04&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=e8b8ffe71e224b9788793ca5938d2513 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:25.927017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48604f-62a2-421c-a69c-2ea20272ac54 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:25.927556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48604f-62a2-421c-a69c-2ea20272ac54 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:25.927675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48604f-62a2-421c-a69c-2ea20272ac54 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:25.929032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1247] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:25.934444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:25.934951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:25.935060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:25.941758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:25.956095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:25.957289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:26.031054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 11:08:26.055998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48604f-62a2-421c-a69c-2ea20272ac54 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.076597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:26.168743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48604f-62a2-421c-a69c-2ea20272ac54 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:26.205556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:26.206186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:26.206464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:26.206733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:26.217399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:26.222096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:26.223183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:26.224137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:26.229614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.332671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49939fb-8b88-453f-aa30-af10dad75404 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:26.364769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:26.370502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:26.371111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:26.371383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:26.371647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:26.372328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:26.372625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:26.373998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:26.382593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:26.388698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.474673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:26.500776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4be0a3-8b67-4760-ac22-98e468eab82d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:26.536594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:26.537217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:26.537479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:26.537734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:26.548619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:26.550056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:26.563846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:26.565040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:26.566312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:26.566312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:26.566312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:26.566312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:26.570830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.683350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cb424-834c-46d4-ab4e-a49258c47876 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:26.723488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:26.724222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:26.724523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:26.724891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:26.725524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:26.725849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:26.727368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:26.740876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:26.746907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.862030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422f7f1-51f6-4a80-bcf9-eafb4bb392bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:26.893620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:26.902796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:26.903527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:26.903717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:26.903982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:26.912576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:26.914149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:26.925929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:26.932303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:26.933438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:26.934038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:26.994875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:27.044062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a8f76-88ae-4cfe-bf14-f87f66ae6386 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:27.080198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:27.080825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:27.081096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:27.081398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:27.082188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:27.082448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:27.083897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:27.102213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:27.108429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:27.223955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe9ac-244b-4fab-8e35-c45bce9d7973 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:27.260594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df843708-959e-4834-8249-35ba695efa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:27.261197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df843708-959e-4834-8249-35ba695efa8d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:27.261461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df843708-959e-4834-8249-35ba695efa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:27.261745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df843708-959e-4834-8249-35ba695efa8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:27.322026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:27.334897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df843708-959e-4834-8249-35ba695efa8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:27.336460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df843708-959e-4834-8249-35ba695efa8d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:27.440612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:08:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:08:27.446782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:27.553782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431d1d3-4974-4d2c-8395-2a03af0914c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:27.569108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:27.593367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:27.593996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:27.594265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:27.594589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:27.608626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:27.610296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:27.626225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:27.627581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:27.628747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:27.628747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:27.628747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:27.628747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:08:27.633850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df843708-959e-4834-8249-35ba695efa8d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:27.738113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:27.751498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df843708-959e-4834-8249-35ba695efa8d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:27.797273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:27.797955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:27.798253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:27.798648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:27.799349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:27.799855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:27.801921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:27.815491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:27.823332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:27.937544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fdaef-6f13-4146-8363-3889a84fa60f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:27.972083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:27.972736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:27.973016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:27.973281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:27.981228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:27.982721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:27.994572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:27.996912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:27.998001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:28.002042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:28.110225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5bb9-d4ef-44ea-bf22-81e2cdea16a2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:28.147148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:28.148018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:28.148344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:28.148700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:28.149461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:28.149813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:28.151908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563ee98-997b-4576-9bbf-b0367a856e9d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:28.166203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:28.172578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:28.200387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:28.283298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437585e-b534-4450-8eac-eeb01387b1ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:28.319013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:28.319602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:28.319920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:28.320185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:28.320917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=9792491cfe3c489a83639e77a27bde04) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:28.321214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:28.322625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e6a5c-c6c5-44c3-951d-285e24e9f998 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:28.327814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:08:28 2022] GET /identity/v3/role_assignments?user.id=9792491cfe3c489a83639e77a27bde04&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:30.332799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbf2c-b0b0-4225-a86b-87a80dbbc735 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:30.333341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbf2c-b0b0-4225-a86b-87a80dbbc735 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:30.333341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacbf2c-b0b0-4225-a86b-87a80dbbc735 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:30.334624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1261] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:30.340238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:30.340755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:30.340755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:30.359624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:30.360853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:30.429923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:30.460859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacbf2c-b0b0-4225-a86b-87a80dbbc735 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:30.477670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:30.585410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacbf2c-b0b0-4225-a86b-87a80dbbc735 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:30.620248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:30.620883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:30.621157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:30.621421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:30.631829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:30.636472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:30.644615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:30.645648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:08:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:30.646543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:30.673027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:30.760014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f73d3-7160-455f-b329-bd9309f1c355 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:30.797970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:30.798564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:30.798837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:30.799102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:30.799762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:30.800044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:30.801626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:30.809839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:08:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:30.816302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:30.893480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:30.931684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd085a8-e101-4bd9-b9d7-31d0c4172caf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:30.969854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:30.970465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:30.970736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:30.970997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:30.981865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:30.983346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:30.999789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:31.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:31.002172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:31.002172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:31.002172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:31.002172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:31.007859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.095745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:31.122305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0ce5-ee77-45a1-9f05-c69dce3f3634 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:31.159993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:31.160549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:31.160802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:31.161061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:31.161698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:31.161988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:31.163346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:31.175398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:31.181343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.290682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e299-ae62-4cfd-a897-e47196be1a29 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:31.310574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:31.325519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:31.326155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:31.326430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:31.326697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:31.334485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:31.336295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:31.348026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:31.349142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:31.350098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:31.355103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.454168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff61c-159a-4e12-95e3-c393a8ed5b27 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:31.490262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:31.490855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:31.491126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:31.491391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:31.492105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:31.492430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:31.493854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:31.502549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:31.511674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:31.517547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.631714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd70a0-2877-415b-9ff1-dc8ab670e0ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:31.669888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:31.670579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:31.670859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:31.671123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:31.685303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:31.687056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:31.703761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:31.704970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:31.706071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:31.706071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:31.706071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:31.706071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:31.711001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.834563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581456-d7a7-4200-9b3d-2662bdde5a72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:31.870314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:31.870903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:31.871174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:31.871441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:31.872067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:31.872381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:31.873745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:31.885942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:31.892002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:31.926471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:31.997637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a511d8-5728-4595-b78f-3a52c286158b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:32.034211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:32.035103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:32.035428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:32.035773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:32.047518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:32.049764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:32.064074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:32.065113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:32.066367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:32.071138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:32.137017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:32.181244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780bd1b-6533-4b40-92a6-7ddbadae65cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:32.216586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:32.217172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:32.217442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:32.217708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:32.218388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:32.218690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:32.220116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e3224-a452-46f9-a9c7-69de63cc930c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:32.232025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:32.238637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:32.341835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:32.351756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827613-e463-4bfa-b32b-48fbc4ec3a05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:32.386229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:32.386855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:32.387124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:32.387388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] PATH_INFO: `/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/9792491cfe3c489a83639e77a27bde04/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:32.389502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b6327b5cd60427eb93d32c22ec52860, user_id=9792491cfe3c489a83639e77a27bde04, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:32.390907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17606ea5-37b3-4978-80af-bda843088c76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:32.407502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 9 11:08:32 2022] PUT /identity/v3/projects/6b6327b5cd60427eb93d32c22ec52860/users/9792491cfe3c489a83639e77a27bde04/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:08:34.387927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095b3-e540-4a6d-b09d-ea4e7b5db032 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:34.388879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095b3-e540-4a6d-b09d-ea4e7b5db032 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:34.388879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017095b3-e540-4a6d-b09d-ea4e7b5db032 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:34.390973 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1274] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:34.396599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:34.397034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:34.397137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:34.416732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:34.417982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:34.487162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:34.510963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017095b3-e540-4a6d-b09d-ea4e7b5db032 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:34.636044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017095b3-e540-4a6d-b09d-ea4e7b5db032 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:34.654710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:34.677710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:34.678283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:34.678554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:34.678821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:34.690529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 11:08:34.695133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:34.696234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:34.697228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 9 11:08:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:34.702574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:34.787061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:34.810107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874799a9-7e7e-478e-8fe7-63f000d2b9d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:34.846923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:34.847520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:34.847794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:34.848065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:34.848758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:34.849057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:34.850410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:34.858355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 9 11:08:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:34.864398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:34.977679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18173b39-7616-44d1-b527-e7c864dca269 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:35.014816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:35.015446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:35.015716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:35.015980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:35.027350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:35.028850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:35.042947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:35.043996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:35.045061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:35.045061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:35.045061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:35.045061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 11:08:35.050600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:35.161066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f072-afa0-4e36-bb51-315f49eca735 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:35.204326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:35.204868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:35.205139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:35.205404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:35.206035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:35.206340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:35.207688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:35.219539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:35.228102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:35.261298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:35.342507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8dddd-f841-4ec0-a97f-66db8e903ab3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:35.377399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:35.378006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:35.378276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:35.378537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:35.386795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:35.388241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 11:08:35.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:35.412350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:35.413554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:35.416101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:35.420056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:35.517836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1690476-6484-4348-ba3a-d27735785ff9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:35.553033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:35.553612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:35.553885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:35.554155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:35.554871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:35.555170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:35.556631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:35.573580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 9 11:08:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:35.579544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:35.676397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:35.688971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa5a8a-92ca-4d74-8612-52bacce70700 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:35.724481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:35.725072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:35.725343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:35.725602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:35.797315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:35.798697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:35.890510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:35.905514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 9 11:08:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 11:08:35.911644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:35.978853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:36.023555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7375a-84e2-4bf7-8297-08ad9e422644 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:36.058047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:36.058586 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:36.058853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:36.059113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:36.069304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:36.070718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 11:08:36.085020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:36.086061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 11:08:36.087100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 11:08:36.087100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 11:08:36.087100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:36.087100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:36.092182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:36.199816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f0f9e-10f3-422a-9236-3da9f65b1463 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:36.238651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:36.239277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:36.239550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:36.239814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:36.240477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:36.240771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:36.242151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:36.253545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 9 11:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:36.262017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:36.380321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc46edc-ba19-4cd4-8aa4-7c39eefa01af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:36.393528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:36.414789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:36.415388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:36.415653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:36.415914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:36.422955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:36.424407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 11:08:36.435449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:36.436489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Aug 09 11:08:36.437719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 9 11:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:36.443158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:36.496415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:36.551082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36e267-6b6e-4239-b00f-2923cf2542ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:36.588989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:36.589592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:36.589864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:36.590126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:36.590794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:36.591095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:36.592546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf959030-c878-415b-9b66-8da5d2fdc9ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:36.604958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 9 11:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:36.611134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:36.722095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5486aaf-3d19-43f6-bc5d-01aec5ed2141 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:36.756696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:36.757275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:36.757541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:36.757804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:36.758511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] RBAC: Adding query filter params (role.id=46b0a9dd588a4810a6ec59469cb69284, scope.project.id=6b6327b5cd60427eb93d32c22ec52860, user.id=9792491cfe3c489a83639e77a27bde04) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:36.758804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:36.760423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a62d7b-d65f-4299-accb-8d5cf7237578 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:36.766564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 9 11:08:36 2022] GET /identity/v3/role_assignments?user.id=9792491cfe3c489a83639e77a27bde04&scope.project.id=6b6327b5cd60427eb93d32c22ec52860&role.id=46b0a9dd588a4810a6ec59469cb69284 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:38.825432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a4f5-5d97-4e43-b87b-cbe4041c7966 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:38.826288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a4f5-5d97-4e43-b87b-cbe4041c7966 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:38.826288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a4f5-5d97-4e43-b87b-cbe4041c7966 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:38.828340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1288] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:38.834538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:38.835298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:38.835444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:38.857161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:38.858349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:38.893393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:38.936039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:38.958530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a4f5-5d97-4e43-b87b-cbe4041c7966 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:39.082953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a4f5-5d97-4e43-b87b-cbe4041c7966 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:39.095422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:39.123574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:39.124185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:39.124487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:39.124748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:39.125318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:39.126676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:39.136013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:39.137249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:39.138546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:08:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:39.146066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:39.248078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4df9e-3778-4447-a8f5-f6bf839fbd6b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:39.283649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:39.284235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:39.284538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:39.284808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:39.285447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:39.285759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:39.287108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d8adb-7600-4a0d-863a-cac73c850c38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:39.292367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 9 11:08:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:39.301513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:39.413185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43347d-0705-4db2-83ce-7e970128074f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:39.448174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:39.448792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:39.449067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:39.449335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:39.449952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:39.450251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:39.451625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50a229-127a-4754-94d9-2c109854011f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:39.456190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 9 11:08:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:41.429779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1441b3-a358-45a1-a3c5-14b7b9a14995 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:41.430374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1441b3-a358-45a1-a3c5-14b7b9a14995 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:41.430374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1441b3-a358-45a1-a3c5-14b7b9a14995 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:41.431640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1293] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:41.438346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231ba8-8ec9-4480-9c1a-e3ab49fd218a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:41.439077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231ba8-8ec9-4480-9c1a-e3ab49fd218a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:41.439260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231ba8-8ec9-4480-9c1a-e3ab49fd218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:41.462875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3231ba8-8ec9-4480-9c1a-e3ab49fd218a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:41.464236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3231ba8-8ec9-4480-9c1a-e3ab49fd218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:41.497912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:41.545625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:41.568707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1441b3-a358-45a1-a3c5-14b7b9a14995 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:41.601324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:41.689698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1441b3-a358-45a1-a3c5-14b7b9a14995 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:41.728840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:41.729720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:41.730028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:41.730418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:41.731074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:41.732862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c38e-5559-40f6-9588-5b19ff3ea423 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:41.748594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 9 11:08:41 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:08:43.727585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66672d9-7a7c-414b-8505-dcd230fc2d6b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:43.728501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66672d9-7a7c-414b-8505-dcd230fc2d6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:43.728501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66672d9-7a7c-414b-8505-dcd230fc2d6b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:43.730040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1296] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:43.736487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:43.737030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:43.737030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:43.764094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:43.765309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:43.848343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:43.871226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66672d9-7a7c-414b-8505-dcd230fc2d6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:43.977259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66672d9-7a7c-414b-8505-dcd230fc2d6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:44.019743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:44.020379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:44.020650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:44.020913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:44.021486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:44.022846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:44.031930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:44.033083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:44.034085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:08:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:44.039362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:44.058227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:44.143824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bded15b-0421-4e8b-974b-429b17faa84a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:44.179231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:44.179820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:44.180091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:44.180383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:44.181002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:44.181293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:44.182803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:44.187891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 9 11:08:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:44.194638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:44.236628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:44.309305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13111049-06f0-412d-b7c4-499723fa011d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:44.343903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:44.344525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:44.344794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:44.345057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:44.345756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] RBAC: Adding query filter params (interface=public, service_id=091a15fe1f7f4617a891b9019f3bf164, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:44.346057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:44.347413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55159eba-e3b9-4357-9111-4d6e6dfac743 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:44.352589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 9 11:08:44 2022] GET /identity/v3/endpoints?service_id=091a15fe1f7f4617a891b9019f3bf164&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:08:44.360092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:44.450350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:44.479830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6673-5ea8-48af-80e4-f6ffef24b152 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:44.512798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:44.513390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:44.513662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:44.513926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:44.514523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:44.515884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea754ef9-8c91-4e3c-bac7-6c8e6b718787 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:44.522873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 9 11:08:44 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:08:46.489482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7179a-4979-4650-9ec7-81d9d806c2ab None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:46.490334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7179a-4979-4650-9ec7-81d9d806c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:46.490334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7179a-4979-4650-9ec7-81d9d806c2ab None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:46.492501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1302] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:08:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:08:46.498646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:46.499333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:46.499490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:46.522428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:08:46.523599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:08:46.594920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:08:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 11:08:46.618061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7179a-4979-4650-9ec7-81d9d806c2ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:46.749472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:46.757132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7179a-4979-4650-9ec7-81d9d806c2ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:46.792120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:46.792771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:46.793042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:46.793310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:46.793891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:46.795277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 11:08:46.804167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 11:08:46.805264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Aug 09 11:08:46.806208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 9 11:08:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 11:08:46.812035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:46.890398 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:46.919262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f26940-6e37-4089-81a2-019a218e7285 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:46.956095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:46.956749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:46.957026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:46.957287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:46.957914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:08:46.958207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:46.959570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ae1eb-ee97-4fcf-8da3-3858be0933c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:46.965013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 9 11:08:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:08:46.971514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:08:47.090098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7dd09-c7a7-406e-82ce-0bf257442713 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:08:47.125333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:08:47.125921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:08:47.126188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:08:47.126449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:08:47.127095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:08:47.128491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b02432-ce69-4ba0-8310-49bd27fe04d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:08:47.151624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:08:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:08:56.961798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=30 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:08:57.147113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:51.163652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340cad-e487-4828-9b06-4620298c8d4a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:51.164277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340cad-e487-4828-9b06-4620298c8d4a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:51.164277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96340cad-e487-4828-9b06-4620298c8d4a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:51.171364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ca5d3-185c-4457-9c40-ec790722534d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:51.171935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ca5d3-185c-4457-9c40-ec790722534d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:51.171935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ca5d3-185c-4457-9c40-ec790722534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:51.176642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1307] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:09:51 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:09:51.269505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd8ca5d3-185c-4457-9c40-ec790722534d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:09:51.271345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd8ca5d3-185c-4457-9c40-ec790722534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:51.443867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:09:51.514214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:51.514758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:51.515165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:51.517619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1309] 10.222.0.27 () {54 vars in 965 bytes} [Tue Aug 9 11:09:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:09:51.524846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115c-8f19-4b85-96e2-1cd9d07892e5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:51.525365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115c-8f19-4b85-96e2-1cd9d07892e5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:51.525957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115c-8f19-4b85-96e2-1cd9d07892e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:51.662269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8030115c-8f19-4b85-96e2-1cd9d07892e5 None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:09:51.665485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8030115c-8f19-4b85-96e2-1cd9d07892e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:51.691535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:51.769278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Aug 9 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:09:51.779334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:09:51.781895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:51.950020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df13c-d7f3-47d0-a1c8-26ccedd8e79d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:09:52.056645 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:09:52.057439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:52.057796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:52.058310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:52.060448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:52.130884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:52.187509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:09:52.189131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66388dd0-5687-450e-acdb-2af00646e3dc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:09:52.210123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:09:54.498855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589129-d189-415b-b147-50d3057625a2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:54.499279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589129-d189-415b-b147-50d3057625a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:54.499372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589129-d189-415b-b147-50d3057625a2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:54.500770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1312] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:09:54.506216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01480239-f342-4b75-969b-ba2bd3023644 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:54.506652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01480239-f342-4b75-969b-ba2bd3023644 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:54.506732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01480239-f342-4b75-969b-ba2bd3023644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:54.590021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-01480239-f342-4b75-969b-ba2bd3023644 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:09:54.591222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01480239-f342-4b75-969b-ba2bd3023644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:54.699314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:09:54.752376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:54.752521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:54.752699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:54.754174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1314] 10.222.0.27 () {54 vars in 965 bytes} [Tue Aug 9 11:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:09:54.760397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd79-0b3c-4c3b-ae91-53d5b2cc80ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:54.760761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd79-0b3c-4c3b-ae91-53d5b2cc80ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:54.761216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855dd79-0b3c-4c3b-ae91-53d5b2cc80ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:54.773797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:54.840061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c855dd79-0b3c-4c3b-ae91-53d5b2cc80ba None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:09:54.841301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c855dd79-0b3c-4c3b-ae91-53d5b2cc80ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:54.924226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1315] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Aug 9 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:09:54.933452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:09:54.935027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:55.042753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7abe15f-192e-43c9-b01d-7b4f42ef7087 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:09:55.116594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:55.119621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:09:55.120298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:55.120585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:55.120886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:55.122995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:55.238026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:09:55.239492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6d644-1d72-4e95-975d-516cebb6deaf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:09:55.260846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:09:58.037719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:58.038297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:58.038297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:58.039891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1317] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:09:58.046594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872c396-306f-4b4d-a309-91af32820085 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:58.047026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872c396-306f-4b4d-a309-91af32820085 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:58.047178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872c396-306f-4b4d-a309-91af32820085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:58.127782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c872c396-306f-4b4d-a309-91af32820085 None None] MFA Rules not processed for user `b554c50e7a13439e8f3d08f498235e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:09:58.129026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c872c396-306f-4b4d-a309-91af32820085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:58.185981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:58.247480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1318] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:09:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:09:58.294576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:09:58.296448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:58.419141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2783e7-3a7b-4ac2-b623-83441734bc16 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:09:58.422053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:09:58.519617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:09:58.520316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:09:58.520660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:09:58.520973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:09:58.522940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:09:58.665947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:09:58.667411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce98e9-3c69-4433-956a-32aad7f0abcd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:09:58.690033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:09:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:01.282352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:01.282937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:01.282937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:01.284297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1320] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:01.290901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86df7-e9b9-44e1-a6c5-855a7c90c0e8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:01.291654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86df7-e9b9-44e1-a6c5-855a7c90c0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:01.291821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d86df7-e9b9-44e1-a6c5-855a7c90c0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:01.336528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:01.373369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a8d86df7-e9b9-44e1-a6c5-855a7c90c0e8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:01.374587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8d86df7-e9b9-44e1-a6c5-855a7c90c0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:01.475375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:01.514934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:01.517196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:01.625771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d89003-5319-4e31-bcc9-085673372f5c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:01.725767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:01.726376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:01.726645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:01.726905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:01.728567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:01.736971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:01.853361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:01.854820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd104e01-8556-41cd-9e07-6df7fc60a883 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:01.873638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:04.667321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96f51-711f-4275-a732-0a961d23422e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:04.667890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96f51-711f-4275-a732-0a961d23422e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:04.667890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96f51-711f-4275-a732-0a961d23422e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:04.669324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1323] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:04.675752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115f13-fdb8-44b3-b37a-1311c42f6c14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:04.676217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115f13-fdb8-44b3-b37a-1311c42f6c14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:04.676338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d115f13-fdb8-44b3-b37a-1311c42f6c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:04.750143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0d115f13-fdb8-44b3-b37a-1311c42f6c14 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:04.751530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d115f13-fdb8-44b3-b37a-1311c42f6c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:04.822559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:04.872766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:04.912751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e96f51-711f-4275-a732-0a961d23422e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:04.914871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05e96f51-711f-4275-a732-0a961d23422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:05.032466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e96f51-711f-4275-a732-0a961d23422e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:05.045650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:05.111237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:05.111893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:05.112168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:05.112472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:05.114143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:05.231605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:05.233011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dca126-e6fc-44e2-9def-cb503e9d8a44 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:05.252533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:07.372633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:07.373207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:07.373207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:07.374497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1326] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:07.380207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291293f-9e17-44c5-9581-4673bf42eee0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:07.380724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291293f-9e17-44c5-9581-4673bf42eee0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:07.380815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291293f-9e17-44c5-9581-4673bf42eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:07.435206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:07.455827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2291293f-9e17-44c5-9581-4673bf42eee0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:07.457101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2291293f-9e17-44c5-9581-4673bf42eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:07.561502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:07.597755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:07.599376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:07.713945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1931c-b79d-4b67-9959-bb08d494b022 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:07.798904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:07.799524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:07.799794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:07.800063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:07.801771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:07.820007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:07.921737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:07.921737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea69d-1206-46a3-950c-f68e73ff8858 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:07.948176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:10.015031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:10.015930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:10.015930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:10.017271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1329] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:10.022761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd707-61c2-4925-a8b6-644e569f654c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:10.023203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd707-61c2-4925-a8b6-644e569f654c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:10.023398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd707-61c2-4925-a8b6-644e569f654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:10.092741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa5dd707-61c2-4925-a8b6-644e569f654c None None] MFA Rules not processed for user `b554c50e7a13439e8f3d08f498235e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:10.094020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa5dd707-61c2-4925-a8b6-644e569f654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:10.163915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:10.201415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:10.241240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:10.242878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:10.348031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a6901-17ca-4c29-94cb-15e0ebab2c40 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:10.368336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:10.427177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:10.427778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:10.428044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:10.428303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:10.429917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:10.552254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:10.552254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb3145-f9ec-44cc-b557-02c1389a6b30 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:10.579335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:12.871777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:12.872359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:12.872359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:12.873874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1332] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:12.879018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae68c1-fa09-4e7e-9792-fe01a851019b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:12.879491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae68c1-fa09-4e7e-9792-fe01a851019b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:12.879609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae68c1-fa09-4e7e-9792-fe01a851019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:12.955473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:12.958428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1ae68c1-fa09-4e7e-9792-fe01a851019b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:12.961083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1ae68c1-fa09-4e7e-9792-fe01a851019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:13.065588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:13.104734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:13.106263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:13.213573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e809d3c-d240-4120-bcfc-d8909f6e171b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:13.290002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:13.290613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:13.290885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:13.291145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:13.292766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:13.320444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:13.413868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:13.415303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8a27a-7a1c-48b2-aae6-6296d8b57494 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:13.442165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:10:15.478405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:15.479017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:15.479017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:15.481072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:15.488000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c66ed-0ede-4c02-9a22-f1fbaebb280a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:15.488499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c66ed-0ede-4c02-9a22-f1fbaebb280a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:15.488607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c66ed-0ede-4c02-9a22-f1fbaebb280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:15.560181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-245c66ed-0ede-4c02-9a22-f1fbaebb280a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:15.561581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-245c66ed-0ede-4c02-9a22-f1fbaebb280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:15.641076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:15.673491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:10:15.717598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:15.720012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:15.839238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e62ce1-ac81-43e3-8fde-e7b8b61aeab9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:15.877722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:15.938282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:15.938910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:15.939177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:15.939437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:15.941068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:16.055515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:16.056970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4424d85-99af-433d-b0b8-9eb2c6ceb875 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:16.075291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:18.482654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:18.483228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:18.483228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:18.484759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1338] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:18.491292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca5529-3cf4-4128-a810-afc00568efb3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:18.492081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca5529-3cf4-4128-a810-afc00568efb3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:18.492227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca5529-3cf4-4128-a810-afc00568efb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:18.579972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-36ca5529-3cf4-4128-a810-afc00568efb3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:18.581236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36ca5529-3cf4-4128-a810-afc00568efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:18.589876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:18.682087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:18.722120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:18.723672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:18.831896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98fce5-02b2-4bfc-b7ac-4e2099af2c2f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:18.911114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:18.911703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:18.911955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:18.912199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:18.913844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:18.952324 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:19.030969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:19.032586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc161f-e1e4-4491-851d-b78883b088c9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:19.051731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:23.168164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:23.168761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:23.168761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:23.170058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1341] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:23.175915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4f18-e31a-4e95-89cc-f8871daaedc7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:23.176445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4f18-e31a-4e95-89cc-f8871daaedc7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:23.176445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4f18-e31a-4e95-89cc-f8871daaedc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:23.245613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-474f4f18-e31a-4e95-89cc-f8871daaedc7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:23.246738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-474f4f18-e31a-4e95-89cc-f8871daaedc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:23.334663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:23.346684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:23.385224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:23.386826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:23.500512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cc9fa-6498-4ea3-aae3-eb39f40a70b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:23.540132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:23.577060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:23.577666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:23.578008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:23.578446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:23.580122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:23.691309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:23.692794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc1af-4f4b-4c9d-86fc-47ca797b21ad service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:23.711063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:26.767136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:26.767675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:26.767675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:26.769252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1344] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:26.774347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da705-8925-4b69-a43b-7c7d5f3436c5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:26.774798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da705-8925-4b69-a43b-7c7d5f3436c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:26.774910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da705-8925-4b69-a43b-7c7d5f3436c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:26.848052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-315da705-8925-4b69-a43b-7c7d5f3436c5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:26.849299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-315da705-8925-4b69-a43b-7c7d5f3436c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:26.868774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:26.957494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:26.996215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:26.997889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:27.105790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39077b-f1bc-49c0-a9d7-3d56c9c7ace4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:27.187181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:27.187799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:27.188081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:27.188372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:27.189975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:27.236611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:27.310768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:27.312298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc324a0f-6678-4a16-a342-d91f19028f3e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:27.331952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:31.414027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d526a2-faf7-423b-8408-660c962325db None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:31.414583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d526a2-faf7-423b-8408-660c962325db None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:31.414583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d526a2-faf7-423b-8408-660c962325db None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:31.417170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1347] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:31.422346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82e94-77a8-4373-bd37-96bf980836d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:31.422807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82e94-77a8-4373-bd37-96bf980836d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:31.422807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b82e94-77a8-4373-bd37-96bf980836d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:31.498422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-31b82e94-77a8-4373-bd37-96bf980836d5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:31.499820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31b82e94-77a8-4373-bd37-96bf980836d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:31.600168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:31.603285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:31.648905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d526a2-faf7-423b-8408-660c962325db None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:31.650609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89d526a2-faf7-423b-8408-660c962325db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:31.765776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d526a2-faf7-423b-8408-660c962325db None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:31.812425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:31.839623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:31.840376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:31.840652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:31.840921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:31.842538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:31.956186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:31.957579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701fab3-c218-4a7f-97db-6ebd2f8218ff service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:31.976736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:36.963279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100cfd2-34f1-46d5-b87c-ee0f4ce99a59 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:36.963794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100cfd2-34f1-46d5-b87c-ee0f4ce99a59 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:36.963794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100cfd2-34f1-46d5-b87c-ee0f4ce99a59 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:36.967192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1350] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:36.972771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d30664-b6f7-4aa8-abfc-8a929bfb7482 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:36.973519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d30664-b6f7-4aa8-abfc-8a929bfb7482 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:36.973925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d30664-b6f7-4aa8-abfc-8a929bfb7482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:37.056534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-89d30664-b6f7-4aa8-abfc-8a929bfb7482 None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:37.058120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89d30664-b6f7-4aa8-abfc-8a929bfb7482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:37.078929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:37.203945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:10:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:41.127371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04ff7-0b87-4c94-bd64-7e96f4c019da None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:41.128343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04ff7-0b87-4c94-bd64-7e96f4c019da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:41.128825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a04ff7-0b87-4c94-bd64-7e96f4c019da None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:41.130266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1352] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:41.135454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc30-9fc5-45e9-99b3-f0b5faf38cbe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:41.135885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc30-9fc5-45e9-99b3-f0b5faf38cbe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:41.135984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ccc30-9fc5-45e9-99b3-f0b5faf38cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:41.214065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-682ccc30-9fc5-45e9-99b3-f0b5faf38cbe None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:41.215267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-682ccc30-9fc5-45e9-99b3-f0b5faf38cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:41.289445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:41.327422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:42.810608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede6fc-7d0c-40f7-8ff2-209955d37dee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:42.811476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede6fc-7d0c-40f7-8ff2-209955d37dee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:42.811476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede6fc-7d0c-40f7-8ff2-209955d37dee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:42.812985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1354] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:42.818364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c57e-4d64-4199-9bf6-aad67c3c244f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:42.818841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c57e-4d64-4199-9bf6-aad67c3c244f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:42.818948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4c57e-4d64-4199-9bf6-aad67c3c244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:42.900077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-18d4c57e-4d64-4199-9bf6-aad67c3c244f None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:42.901344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18d4c57e-4d64-4199-9bf6-aad67c3c244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:43.011840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:46.412853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8bb5-ecce-441f-a6ec-613e0e8db03e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:46.413717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8bb5-ecce-441f-a6ec-613e0e8db03e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:46.414011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a8bb5-ecce-441f-a6ec-613e0e8db03e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:46.415532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1356] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:46.422969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69689bd2-fc75-41db-8852-3421d7be9e11 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:46.423912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69689bd2-fc75-41db-8852-3421d7be9e11 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:46.424474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69689bd2-fc75-41db-8852-3421d7be9e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:46.467205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:46.500506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-69689bd2-fc75-41db-8852-3421d7be9e11 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:46.501698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69689bd2-fc75-41db-8852-3421d7be9e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:46.605727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:46.661491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:46.662100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:46.662676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:46.665761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1358] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 9 11:10:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:46.675138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41fe9-341e-469b-8c19-735f538692f8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:46.675506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41fe9-341e-469b-8c19-735f538692f8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:46.675885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c41fe9-341e-469b-8c19-735f538692f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:46.756247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37c41fe9-341e-469b-8c19-735f538692f8 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:46.757475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37c41fe9-341e-469b-8c19-735f538692f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:46.809502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:46.884176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 9 11:10:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:46.892658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:46.894443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:47.045844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee15f9-1a8a-42d7-ad6c-1986ba7bd648 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:47.120526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:47.162853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:47.163375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:47.163651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:47.163872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:47.165484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:47.276500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:47.278209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5bc1e-e0b4-4ca8-9234-80cd872b5e7b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:47.297791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:49.574130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:49.574677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:49.574677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:49.580479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1361] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:49.582089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d16d79-1800-4daf-bdd1-764e99f46469 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:49.582992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d16d79-1800-4daf-bdd1-764e99f46469 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:49.584040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d16d79-1800-4daf-bdd1-764e99f46469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:49.655890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-51d16d79-1800-4daf-bdd1-764e99f46469 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:49.657108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51d16d79-1800-4daf-bdd1-764e99f46469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:49.698351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:49.761973 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:49.809592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:49.811357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:49.987821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f274f4-d3a1-493e-9a51-7cfd00492c3e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:49.998292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:50.109973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:50.110584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:50.110866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:50.111102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:50.112894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:50.219167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:50.222109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d6c48-ea17-4cad-b608-5f9b185e6d0a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:50.242333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:52.730736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be483f0-d89d-455b-80f5-9d7ab5abd4c7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:52.731379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be483f0-d89d-455b-80f5-9d7ab5abd4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:52.731379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be483f0-d89d-455b-80f5-9d7ab5abd4c7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:52.733087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1364] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:52.740947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45873f1b-7ff8-4606-a334-3212276770f7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:52.740947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45873f1b-7ff8-4606-a334-3212276770f7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:52.740947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45873f1b-7ff8-4606-a334-3212276770f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:52.755572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:52.820220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-45873f1b-7ff8-4606-a334-3212276770f7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:52.821452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45873f1b-7ff8-4606-a334-3212276770f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:52.932342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:52.974722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:52.974999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:52.975467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:52.976862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1366] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 9 11:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:52.982587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad039-ceaa-44ef-a33d-474da74ed730 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:52.982813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad039-ceaa-44ef-a33d-474da74ed730 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:52.983074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad039-ceaa-44ef-a33d-474da74ed730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:53.061770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c0ad039-ceaa-44ef-a33d-474da74ed730 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:53.062969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c0ad039-ceaa-44ef-a33d-474da74ed730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:53.109549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:53.207622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 9 11:10:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:53.215593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:53.217465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:53.374704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b37f4-4c8d-4554-9a58-439cd5270444 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:53.395957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:53.494955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:53.495570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:53.495844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:53.496107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:53.497800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:53.595444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:53.597118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3424f-fc2c-426d-aa2a-ca11916df027 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:53.617387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:54.483108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2872b4-100a-4cde-84ce-95e53b99e7f2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:54.483673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2872b4-100a-4cde-84ce-95e53b99e7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:54.483673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2872b4-100a-4cde-84ce-95e53b99e7f2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:54.485158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1369] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:54.490301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c68c-6eee-4766-84f8-c2f346413bb6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:54.490744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c68c-6eee-4766-84f8-c2f346413bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:54.490826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1c68c-6eee-4766-84f8-c2f346413bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:54.525891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:54.566031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-44b1c68c-6eee-4766-84f8-c2f346413bb6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:54.567224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44b1c68c-6eee-4766-84f8-c2f346413bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:54.667462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:55.010920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:55.010920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:55.010920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:55.014686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1371] 10.222.0.27 () {54 vars in 964 bytes} [Tue Aug 9 11:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:55.017879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74f51c-9832-428d-919e-9f413948223c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:55.018132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74f51c-9832-428d-919e-9f413948223c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:55.018419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74f51c-9832-428d-919e-9f413948223c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:55.097327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa74f51c-9832-428d-919e-9f413948223c None None] MFA Rules not processed for user `befb4776ff7b4014997d32e4697f29a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:55.098642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa74f51c-9832-428d-919e-9f413948223c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:55.152830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:55.185462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Tue Aug 9 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:10:55.193934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:55.195533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:55.306199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e544d1b-0db7-489f-93e5-d202910c395e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:55.354451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:55.393380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:55.393996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:55.394273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:55.394539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:55.396185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:55.516509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:55.517956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d55566-825d-4f36-857c-0304e6520416 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:55.537365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:55.714269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:55.716145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:55.716375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:55.720814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1374] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:55.723732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c3a6-3f6f-47d0-80f2-c23ed0d16603 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:55.724049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c3a6-3f6f-47d0-80f2-c23ed0d16603 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:55.724429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17c3a6-3f6f-47d0-80f2-c23ed0d16603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:55.792462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf17c3a6-3f6f-47d0-80f2-c23ed0d16603 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:55.793681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf17c3a6-3f6f-47d0-80f2-c23ed0d16603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:55.815596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:55.904725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:55.938064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:55.939964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:56.090704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc8604-8168-4a71-9841-bb73f34d5858 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:56.155639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:56.221558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:56.222167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:56.222448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:56.222702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:56.224363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:56.316176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:56.317829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08723ad-5ee8-449b-a325-1e1453129780 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:56.336293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:10:58.363886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:58.364485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:58.365018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:58.366289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1377] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:10:58.371796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d410-bae9-4e26-817c-0bd652fe9a69 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:58.372297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d410-bae9-4e26-817c-0bd652fe9a69 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:58.372600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0d410-bae9-4e26-817c-0bd652fe9a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:58.451036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9d0d410-bae9-4e26-817c-0bd652fe9a69 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:10:58.452230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9d0d410-bae9-4e26-817c-0bd652fe9a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:58.461004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:58.570175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:10:58.615188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:10:58.616677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:58.777105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27aee10-8e79-43da-9531-a5296b416fbd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:10:58.781496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:10:58.910279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:10:58.910992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:10:58.911293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:10:58.911654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:10:58.913358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:10:59.033259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:10:59.035067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9c06-1dce-441c-a9c7-fecb8329cea4 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:10:59.057747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:01.118903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:01.119462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:01.119462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:01.120975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1380] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:01.126433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a201-2344-4df5-918c-023a4db6d6bf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:01.126911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a201-2344-4df5-918c-023a4db6d6bf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:01.126911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a201-2344-4df5-918c-023a4db6d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:01.140130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:01.198098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4b7a201-2344-4df5-918c-023a4db6d6bf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:01.199184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4b7a201-2344-4df5-918c-023a4db6d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:01.304542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:01.348989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:01.351768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:01.502513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:01.514620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778f9a9-1255-41b3-b2d1-6ac30ccdfaf3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:01.635012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:01.635839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:01.636204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:01.636608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:01.638725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:01.735527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:01.737353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ee260-c56b-426a-8a0d-c9406e808d61 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:01.748656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:01.758264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:03.850825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:03.851544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:03.851544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:03.852815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1383] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:03.858380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7781-fece-4bac-b0e1-3972e2ac0ae9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:03.858833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7781-fece-4bac-b0e1-3972e2ac0ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:03.858944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b7781-fece-4bac-b0e1-3972e2ac0ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:03.929104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ea4b7781-fece-4bac-b0e1-3972e2ac0ae9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:03.930330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea4b7781-fece-4bac-b0e1-3972e2ac0ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:04.040082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:04.085313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:04.087234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:04.186879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:04.269277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97f37c-f3b0-4e23-8704-5e093ba3cca0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:04.393739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:04.394357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:04.394629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:04.394896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:04.396538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:04.460340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:04.505269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:04.506883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bb6e2-8fb9-48bc-a773-63e833fbe43d service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:04.526057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:06.590195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:06.590780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:06.591058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:06.592313 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1386] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:06.597993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164b9a-0252-40c8-9097-bb2e7bfe0142 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:06.598490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164b9a-0252-40c8-9097-bb2e7bfe0142 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:06.598587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164b9a-0252-40c8-9097-bb2e7bfe0142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:06.679294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9164b9a-0252-40c8-9097-bb2e7bfe0142 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:06.680607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9164b9a-0252-40c8-9097-bb2e7bfe0142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:06.785818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:06.832060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:06.833676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:06.933844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:07.025244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e848b45-10f7-4613-86fc-92df56763048 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:07.152949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:07.153660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:07.153932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:07.154193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:07.155807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:07.194693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:07.251186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:07.253453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66045882-5831-4bcd-905a-4d5b1d0770dd service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:07.278147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:09.356992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:09.357579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:09.357579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:09.359026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1389] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:09.364134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e997435-9637-403b-9693-380ff81f042b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:09.364573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e997435-9637-403b-9693-380ff81f042b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:09.364679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e997435-9637-403b-9693-380ff81f042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:09.431653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e997435-9637-403b-9693-380ff81f042b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:09.432904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e997435-9637-403b-9693-380ff81f042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:09.532684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:09.577583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:09.579186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:09.610675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:09.728242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc9a39-de56-4e4a-a714-73aa9942234c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:09.832748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:09.844941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:09.845555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:09.845806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:09.846064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:09.847634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:09.941786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:09.943424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202a4e7-188f-4e7e-9498-1011768f3933 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:09.961551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:10.798078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9155f-3366-4003-845f-38828667562b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:10.798600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9155f-3366-4003-845f-38828667562b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:10.798600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9155f-3366-4003-845f-38828667562b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:10.799939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1392] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:10.805232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88046df-8280-425b-ba8d-7168da1b6f12 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:10.805689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88046df-8280-425b-ba8d-7168da1b6f12 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:10.805795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88046df-8280-425b-ba8d-7168da1b6f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:10.871980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b88046df-8280-425b-ba8d-7168da1b6f12 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:10.873203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b88046df-8280-425b-ba8d-7168da1b6f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:10.933276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:10.972243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1393] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:11.030860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:11.031144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:11.031399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:11.032843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1394] 10.222.0.27 () {54 vars in 967 bytes} [Tue Aug 9 11:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:11.038115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a367b-42d1-4b97-9e0a-44f3a6fa9fd3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:11.038399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a367b-42d1-4b97-9e0a-44f3a6fa9fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:11.039327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a367b-42d1-4b97-9e0a-44f3a6fa9fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:11.125662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da0a367b-42d1-4b97-9e0a-44f3a6fa9fd3 None None] MFA Rules not processed for user `32cbfe0d7b584729b45b85c65377648f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:11.126870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da0a367b-42d1-4b97-9e0a-44f3a6fa9fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:11.210298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1395] 10.222.0.27 () {58 vars in 1064 bytes} [Tue Aug 9 11:11:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:11.218356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:11.219843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:11.234842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:11.327856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfdb78-b3fe-44d5-ba04-9a6357fb27ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:11.403269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:11.403868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:11.404138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:11.404415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:11.405998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:11.485392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:11.530453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:11.531904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10285abf-5101-45d9-b4e4-6860a9c1a710 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:11.550045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:12.013873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.014405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.014405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.015762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1397] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:12.021277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b0cd-2e52-424e-ae6f-e66b24f751b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.021756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b0cd-2e52-424e-ae6f-e66b24f751b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.021850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941b0cd-2e52-424e-ae6f-e66b24f751b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.090622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d941b0cd-2e52-424e-ae6f-e66b24f751b6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:12.091864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d941b0cd-2e52-424e-ae6f-e66b24f751b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.203987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1398] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:12.250910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:12.252531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.290179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c2f6-2876-448a-9ded-b80b5e677ab6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.290665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c2f6-2876-448a-9ded-b80b5e677ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.290779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c2f6-2876-448a-9ded-b80b5e677ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.317022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:12.321120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:12.357951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f849c2f6-2876-448a-9ded-b80b5e677ab6 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:12.359155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f849c2f6-2876-448a-9ded-b80b5e677ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.404853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d4cc7-6bb3-4873-9d83-f4245ad14971 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:12.444477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1399] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:12.486680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eab1f-8343-4268-a433-6fe1faba1694 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.487044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eab1f-8343-4268-a433-6fe1faba1694 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.487381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eab1f-8343-4268-a433-6fe1faba1694 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.489003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1400] 10.222.0.27 () {54 vars in 943 bytes} [Tue Aug 9 11:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:11:12.494905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c970b-4b51-4b04-b0ca-70e0af70a92e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.495178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c970b-4b51-4b04-b0ca-70e0af70a92e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.495446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c970b-4b51-4b04-b0ca-70e0af70a92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.521797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:12.522405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.522681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.522949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.524573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.545631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:12.572090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd9c970b-4b51-4b04-b0ca-70e0af70a92e None None] MFA Rules not processed for user `f55820319ba442aa8288cf179681b2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:12.573544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd9c970b-4b51-4b04-b0ca-70e0af70a92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.614037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:12.615646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:12.628533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:12.634251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1401] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:12.689773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1402] 10.222.0.27 () {58 vars in 1040 bytes} [Tue Aug 9 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:11:12.700340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:12.701845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:12.819576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d34cf-ab4a-4cd5-a0e3-3f65da99b2f8 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:12.882730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:12.936456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:12.937066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:12.937334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:12.937601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:12.939261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:13.026490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:13.028261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154f8b1-240a-460f-a3b9-3c3a47a7a8d0 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:13.037005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1403] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:11:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:11:14.712153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:14.712153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:14.712153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:14.712153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1404] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:14.729578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1517-e263-44c6-b15f-9cbd8c9828c9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:14.729578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1517-e263-44c6-b15f-9cbd8c9828c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:14.729578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1517-e263-44c6-b15f-9cbd8c9828c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:14.783402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-356e1517-e263-44c6-b15f-9cbd8c9828c9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:14.784663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-356e1517-e263-44c6-b15f-9cbd8c9828c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:14.827730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:14.891309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:14.935171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:14.936740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:15.105217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7d82e-7966-4928-a77d-f11a1f5b03bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:15.140819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:15.243845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:15.244874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:15.245342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:15.245602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:15.248033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:15.357056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:15.358618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f60592-f4c9-45b5-aed2-c1870e27aa73 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:15.381117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:17.441893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:17.442501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:17.442501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:17.444320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1407] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:17.449431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388771-bc27-4053-a79f-e0282f2451cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:17.449983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388771-bc27-4053-a79f-e0282f2451cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:17.449983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca388771-bc27-4053-a79f-e0282f2451cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:17.491608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:17.520536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca388771-bc27-4053-a79f-e0282f2451cc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:17.521748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca388771-bc27-4053-a79f-e0282f2451cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:17.634701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:17.679712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:17.681408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:17.834572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31e5c4-ab5e-4fd0-86a5-474559f10871 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:17.863400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:17.967824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:17.968465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:17.968736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:17.969018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:17.970642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:18.065234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:18.066857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1261ae-27e8-40b3-ae2d-6f56bbe2095b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:18.089246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:27.633201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=51 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:28.086238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:34.862956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6786d8-baf3-45c3-a43e-2477e7466377 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:34.863545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6786d8-baf3-45c3-a43e-2477e7466377 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:34.863545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6786d8-baf3-45c3-a43e-2477e7466377 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:34.865507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1410] 10.222.0.27 () {54 vars in 918 bytes} [Tue Aug 9 11:11:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:34.873124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7da61-65df-4478-9cfe-3d7f3b8c0d57 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:34.873597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7da61-65df-4478-9cfe-3d7f3b8c0d57 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:34.873698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7da61-65df-4478-9cfe-3d7f3b8c0d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:35.082222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-27c7da61-65df-4478-9cfe-3d7f3b8c0d57 None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:35.083377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-27c7da61-65df-4478-9cfe-3d7f3b8c0d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:35.210959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Aug 9 11:11:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:35.233970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a0d38-0c9b-438c-863e-7c232bd111c4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:35.234249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a0d38-0c9b-438c-863e-7c232bd111c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:35.234583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a0d38-0c9b-438c-863e-7c232bd111c4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:35.235995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1412] 10.222.0.27 () {54 vars in 918 bytes} [Tue Aug 9 11:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:35.242096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b3ee-8987-47d3-a661-d6d5c4136173 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:35.242430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b3ee-8987-47d3-a661-d6d5c4136173 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:35.242785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04b3ee-8987-47d3-a661-d6d5c4136173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:35.326221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:35.329081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb04b3ee-8987-47d3-a661-d6d5c4136173 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:35.331026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb04b3ee-8987-47d3-a661-d6d5c4136173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:35.454149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Aug 9 11:11:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:37.923538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:37.924066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:37.924066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:37.925448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:37.931506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853ea0-84f8-41a7-b914-8348245ab4ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:37.932146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853ea0-84f8-41a7-b914-8348245ab4ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:37.932320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853ea0-84f8-41a7-b914-8348245ab4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:38.017534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50853ea0-84f8-41a7-b914-8348245ab4ee None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:38.018722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50853ea0-84f8-41a7-b914-8348245ab4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:38.081367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:38.124069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:38.166672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:38.168800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:38.321396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085cc9f-a988-40d4-abcf-19ac23989d72 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:38.401085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:38.441900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:38.442427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:38.442692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:38.442951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:38.444719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:38.544120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:38.545782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1d63e-55ec-477f-939f-a733e3d3a27a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:38.564352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:39.470896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:39.471836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:39.471836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:39.473980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:39.481880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094c59-52b2-4491-bcae-12c846529cf4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:39.482304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094c59-52b2-4491-bcae-12c846529cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:39.482423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66094c59-52b2-4491-bcae-12c846529cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:39.560734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66094c59-52b2-4491-bcae-12c846529cf4 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:39.561922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66094c59-52b2-4491-bcae-12c846529cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:39.590331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:39.668618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:11:39.821063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:39.822577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:39.978804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782e21a-dbab-4589-b444-34cdfe105f89 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:40.021323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:40.103324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:40.103839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:40.104103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:40.104393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:40.106011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:40.201744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:40.203320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee733-f963-4c60-b110-e710d1024e70 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:40.221445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:41.098456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:41.099012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:41.099012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:41.100254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:41.105845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c699-8037-4867-8ec4-01ad5c29c191 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:41.106295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c699-8037-4867-8ec4-01ad5c29c191 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:41.106382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c699-8037-4867-8ec4-01ad5c29c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:41.163338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:41.187975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c742c699-8037-4867-8ec4-01ad5c29c191 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:41.189225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c742c699-8037-4867-8ec4-01ad5c29c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:41.293432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:41.452252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:41.455454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:41.622821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97833442-73f3-4738-9c56-dc1ac61fb362 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:41.635115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:41.741262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:41.741773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:41.742045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:41.742308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:41.743954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:41.840202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:41.841860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0333e-8128-49c1-ad02-5fbe75809863 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:41.861376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:42.727855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:42.728499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:42.728499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:42.729748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:42.734909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcce08-e2ef-400f-9ec9-f161b4f2b9c3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:42.735399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcce08-e2ef-400f-9ec9-f161b4f2b9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:42.735399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcce08-e2ef-400f-9ec9-f161b4f2b9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:42.760258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:42.819300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1dcce08-e2ef-400f-9ec9-f161b4f2b9c3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:42.820569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1dcce08-e2ef-400f-9ec9-f161b4f2b9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:42.933034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:43.083839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:43.086360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:43.213834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:43.263304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0cb93-f6b6-4cf6-990e-ad5d312f398e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:43.383051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:43.383570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:43.383838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:43.384098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:43.385707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:43.469594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:43.485300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:43.486926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489cb2a-729a-4c1d-a17e-ff7f3612a00a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:43.506537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:44.364921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:44.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:44.365783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:44.367850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:44.374760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663582-2b0d-4dd0-ad49-07d3d282888d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:44.375185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663582-2b0d-4dd0-ad49-07d3d282888d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:44.375327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663582-2b0d-4dd0-ad49-07d3d282888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:44.472044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e6663582-2b0d-4dd0-ad49-07d3d282888d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:44.473315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6663582-2b0d-4dd0-ad49-07d3d282888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:44.581070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:44.734176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:44.736242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:44.838047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:44.910049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bb54e-dba5-4c98-b9ed-4596b2fb0297 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:45.030783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:45.031393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:45.031664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:45.031925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:45.033580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:45.085034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:45.129702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:45.131368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcde15-3019-4dcc-8e82-26cbc72684b7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:45.151122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:45.957135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78214e-9391-4958-839e-f8409874f8db None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:45.957636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78214e-9391-4958-839e-f8409874f8db None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:45.957636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78214e-9391-4958-839e-f8409874f8db None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:45.959009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:45.964211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a463d1-12bc-4ca6-80df-0efc8d73f603 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:45.964783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a463d1-12bc-4ca6-80df-0efc8d73f603 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:45.964783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a463d1-12bc-4ca6-80df-0efc8d73f603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:46.051133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94a463d1-12bc-4ca6-80df-0efc8d73f603 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:46.052763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94a463d1-12bc-4ca6-80df-0efc8d73f603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:46.164100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:46.318153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78214e-9391-4958-839e-f8409874f8db None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:46.320190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe78214e-9391-4958-839e-f8409874f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:46.371600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:46.478212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78214e-9391-4958-839e-f8409874f8db None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:46.612170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:46.612829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:46.613100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:46.613363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:46.614971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:46.622278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:46.709328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:46.710926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568e4b0-bc71-405f-ab3b-d717282e0f4e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:46.730420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:47.540929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:47.541571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:47.541571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:47.543427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:47.549390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3c3d-84ba-4f5b-a8cb-b9531d703f1b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:47.549854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3c3d-84ba-4f5b-a8cb-b9531d703f1b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:47.549982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f3c3d-84ba-4f5b-a8cb-b9531d703f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:47.637037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-588f3c3d-84ba-4f5b-a8cb-b9531d703f1b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:47.638256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-588f3c3d-84ba-4f5b-a8cb-b9531d703f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:47.724406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:47.755413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:47.914967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:47.917845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:47.941509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:48.089895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d3f7-721d-40fc-a8ff-b13ba24bc999 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:48.179503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:48.207951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:48.208612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:48.208912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:48.209329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:48.211035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:48.305704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:48.307315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c1cd0-b69a-47c7-8f0d-17c228e8153e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:48.326138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:11:49.152544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:49.153128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:49.153128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:49.154361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:49.160752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19c5f-2a2f-47f0-996c-d0ec900bb67e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:49.161244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19c5f-2a2f-47f0-996c-d0ec900bb67e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:49.161244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc19c5f-2a2f-47f0-996c-d0ec900bb67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:49.248280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0bc19c5f-2a2f-47f0-996c-d0ec900bb67e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:49.249513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0bc19c5f-2a2f-47f0-996c-d0ec900bb67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:49.294896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:49.353068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:49.516981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:49.518501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:49.677695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bb6c8-ad12-4e4f-b6dc-f6ebd8011188 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:49.744087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:49.804409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:49.804957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:49.805228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:49.805485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:49.807079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:49.911069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:49.912861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c7e00-e9f1-4b89-bc67-04bfc0d5c2da service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:49.933928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:50.741126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:50.741739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:50.741739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:50.743018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:50.749528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eff5f-8310-4775-b28b-12e89e146b03 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:50.750209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eff5f-8310-4775-b28b-12e89e146b03 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:50.750389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eff5f-8310-4775-b28b-12e89e146b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:50.859217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f6eff5f-8310-4775-b28b-12e89e146b03 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:50.860459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f6eff5f-8310-4775-b28b-12e89e146b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:50.862544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:50.983934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:51.186083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:51.188408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:51.380097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276d414-289d-4b26-9a94-a882a2e4e9bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:51.418945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:51.541255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:51.541877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:51.542236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:51.542536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:51.544185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:51.656181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:51.657796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21aded-80bc-4d3f-8c72-a347b66eb580 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:51.677216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:53.578196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:53.579063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:53.579063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:53.581219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:11:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:53.586869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e621-244e-4180-a054-25b78809fbae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:53.587314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e621-244e-4180-a054-25b78809fbae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:53.587419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e621-244e-4180-a054-25b78809fbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:53.629330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:53.655034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b717e621-244e-4180-a054-25b78809fbae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:53.656233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b717e621-244e-4180-a054-25b78809fbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:53.760933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:11:53.803522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:53.806012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:53.924129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5cdb3-2854-4434-b48e-b90211c18e30 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:53.958935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:54.004077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:54.004727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:54.004996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:54.005259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:54.006834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:54.122693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:54.124114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671f5e2-bd27-489f-ac3c-ed4a2214095a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:54.144090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:56.332404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:56.333004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:56.333004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:56.334293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:56.340684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1e3c-8958-42e0-ae05-9a71eeb715e2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:56.341165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1e3c-8958-42e0-ae05-9a71eeb715e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:56.341297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1e3c-8958-42e0-ae05-9a71eeb715e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:56.422998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-79fd1e3c-8958-42e0-ae05-9a71eeb715e2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:56.424215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79fd1e3c-8958-42e0-ae05-9a71eeb715e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:56.437408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:56.523166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:56.564291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:56.566623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:56.684718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27acddf7-5d86-4356-beeb-14668964f3e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:56.697440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:56.762441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:56.763048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:56.763317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:56.763580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:56.765274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:56.888566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:56.890009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b15da-e1be-4879-b68f-5203051f374b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:56.908102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:11:59.261480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:59.262421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:59.262421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:59.264684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:11:59.273763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c77cf2-bc9e-480d-b3cb-3fbe544d3401 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:59.274530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c77cf2-bc9e-480d-b3cb-3fbe544d3401 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:59.274698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c77cf2-bc9e-480d-b3cb-3fbe544d3401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:59.328026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:59.348058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0c77cf2-bc9e-480d-b3cb-3fbe544d3401 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:11:59.349309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0c77cf2-bc9e-480d-b3cb-3fbe544d3401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:59.459524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:11:59.505484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:11:59.507722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:59.623798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0fb36-4223-443e-96d8-40867bf57df5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:11:59.676852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:11:59.710658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:11:59.711250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:11:59.711507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:11:59.711773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:11:59.713392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:11:59.837154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:11:59.838633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dea25b-bded-4a3c-bff2-b730e4678aff service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:11:59.858016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:01.986972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:01.987841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:01.987841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:01.990052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:01.996415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f524ab-e07f-499e-b4cd-8f805405336e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:01.997131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f524ab-e07f-499e-b4cd-8f805405336e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:01.997359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f524ab-e07f-499e-b4cd-8f805405336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:02.071984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34f524ab-e07f-499e-b4cd-8f805405336e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:02.073274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34f524ab-e07f-499e-b4cd-8f805405336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:02.096098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:02.182703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:02.228178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:02.230516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:02.348882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b89f1-21b0-4544-b83c-02a7ece25a66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:02.372233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:02.428372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:02.428967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:02.429233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:02.429490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:02.431070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:02.561243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:02.562693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aed279-cc80-47de-89ae-f1778761c0dc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:02.581766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:04.615203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:04.616049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:04.616049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:04.618011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:04.624951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4580f6-1265-47a1-a9d5-aa75cda1b00e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:04.625383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4580f6-1265-47a1-a9d5-aa75cda1b00e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:04.625508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4580f6-1265-47a1-a9d5-aa75cda1b00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:04.695548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:04.698825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3f4580f6-1265-47a1-a9d5-aa75cda1b00e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:04.701060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f4580f6-1265-47a1-a9d5-aa75cda1b00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:04.806431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:04.857213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:04.858804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:04.968179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918d96f-952f-4f1a-939e-710278f5f978 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:05.018937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:05.046451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:05.047054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:05.047334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:05.047598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:05.049227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:05.166244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:05.167816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568113c0-189e-42ff-bd9d-2c2e282793fd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:05.196068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:07.264012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:07.264628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:07.264628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:07.266255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:07.273452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4633c-64b3-4ab6-b728-048ed613609b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:07.273922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4633c-64b3-4ab6-b728-048ed613609b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:07.274036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4633c-64b3-4ab6-b728-048ed613609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:07.350811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-67b4633c-64b3-4ab6-b728-048ed613609b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:07.352025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67b4633c-64b3-4ab6-b728-048ed613609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:07.381947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:07.450949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:07.496827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:07.499228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:07.632151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c7656-4258-4415-a9c0-c18e6ead3c48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:07.667142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:07.719936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:07.720588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:07.720866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:07.721129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:07.722761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:07.837198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:07.838806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be7366-2b79-4c3f-9f47-3dce68bcc6dc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:07.857860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:09.893982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:09.894611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:09.894611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:09.895849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 11:12:09.902137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba942b-02a5-4d2d-a90f-d1c0398cbe90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:09.902583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba942b-02a5-4d2d-a90f-d1c0398cbe90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:09.902697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba942b-02a5-4d2d-a90f-d1c0398cbe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:09.973430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41ba942b-02a5-4d2d-a90f-d1c0398cbe90 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:09.974681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41ba942b-02a5-4d2d-a90f-d1c0398cbe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:09.983207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:10.074569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:10.117566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:10.120429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:10.245423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec38cc4-6659-48c9-b2e9-e7da6f38fb30 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:10.310458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:10.329877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:10.330482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:10.330757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:10.331021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:10.332666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:10.449228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:10.450646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aa02c-825d-4cad-a9c7-c8b0a55796da service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:10.470571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:12.580759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:12.581638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:12.581638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:12.583715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:12.592730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231171-46ba-4fa1-8f57-b886cac20d72 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:12.593155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231171-46ba-4fa1-8f57-b886cac20d72 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:12.593311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21231171-46ba-4fa1-8f57-b886cac20d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:12.668078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-21231171-46ba-4fa1-8f57-b886cac20d72 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:12.669316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-21231171-46ba-4fa1-8f57-b886cac20d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:12.715120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:12.786380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:12.824134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:12.825735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:12.934436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eabe7d-2695-4f16-b805-220c86095ac8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:12.974791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:13.014412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:13.015018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:13.015292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:13.015559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:13.017201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:13.134723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:13.136875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9570c-069e-461f-b303-17736722f5b6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:13.156147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:15.177596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:15.178172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:15.178172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:15.179411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:15.186036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc74a-c76c-4823-8aff-c88581c63fb5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:15.186683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc74a-c76c-4823-8aff-c88581c63fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:15.186802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc74a-c76c-4823-8aff-c88581c63fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:15.274151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ebbc74a-c76c-4823-8aff-c88581c63fb5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:15.275351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ebbc74a-c76c-4823-8aff-c88581c63fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:15.294960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:15.390128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:15.439157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:15.440754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:15.557286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963026d-1ef4-4970-893f-fddf629d39aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:15.628611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:15.636505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:15.637114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:15.637413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:15.637651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:15.639281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:15.754315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:15.755746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac5a36-c4a3-4a43-9350-b53afeba6cda service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:15.774701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:17.820016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:17.820749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:17.820749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:17.821954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:17.828428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d6bf-ee14-436d-b441-8b3184e97da5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:17.828889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d6bf-ee14-436d-b441-8b3184e97da5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:17.829002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d6bf-ee14-436d-b441-8b3184e97da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:17.910300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8674d6bf-ee14-436d-b441-8b3184e97da5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:17.911516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8674d6bf-ee14-436d-b441-8b3184e97da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:17.960607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:18.011123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:18.048741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:18.050316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:18.175449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0d91-89ce-47bf-a597-6f3dd74ec915 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:18.223526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:18.250259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:18.250866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:18.251137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:18.251400 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:18.253058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:18.369121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:18.370596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc34f-05a7-4f6f-80b0-78939962d33b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:18.390723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:28.009102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=22 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:28.388257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:37.764939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2f6c-b3b6-4c17-a641-c46454fdb5da None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:37.765536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2f6c-b3b6-4c17-a641-c46454fdb5da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:37.765536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb2f6c-b3b6-4c17-a641-c46454fdb5da None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:37.766814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1471] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:37.772230 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf4e12-591c-4ae3-98fd-f91f7c17fb24 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:37.772704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf4e12-591c-4ae3-98fd-f91f7c17fb24 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:37.772798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf4e12-591c-4ae3-98fd-f91f7c17fb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:37.842866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ffaf4e12-591c-4ae3-98fd-f91f7c17fb24 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:37.843966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ffaf4e12-591c-4ae3-98fd-f91f7c17fb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:37.953446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:38.050631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:38.050938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:38.051210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:38.052652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1473] 10.222.0.27 () {54 vars in 967 bytes} [Tue Aug 9 11:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:38.057720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bafb5a-2212-4ed0-9768-9190efd473f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:38.057996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bafb5a-2212-4ed0-9768-9190efd473f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:38.058266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bafb5a-2212-4ed0-9768-9190efd473f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:38.141057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-62bafb5a-2212-4ed0-9768-9190efd473f5 None None] MFA Rules not processed for user `32cbfe0d7b584729b45b85c65377648f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:38.142172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-62bafb5a-2212-4ed0-9768-9190efd473f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:38.165792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:38.227233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1474] 10.222.0.27 () {58 vars in 1064 bytes} [Tue Aug 9 11:12:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:38.235163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:38.236752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:38.349750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0273d-04d1-40a1-8dd5-5717fea2067b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:38.424103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:38.424648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:38.424921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:38.425191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:38.426773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:38.496045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:38.556927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:38.558467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310d713-2164-47eb-9467-174a2f84a056 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:38.578447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:40.650323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:40.651133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:40.651133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:40.652417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1476] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:40.657891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b69c-c00d-4792-ac1e-e0f2045f36ae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:40.658349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b69c-c00d-4792-ac1e-e0f2045f36ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:40.658450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b69c-c00d-4792-ac1e-e0f2045f36ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:40.733343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-48f7b69c-c00d-4792-ac1e-e0f2045f36ae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:40.734553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48f7b69c-c00d-4792-ac1e-e0f2045f36ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:40.839757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:12:40.862250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:40.863788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:41.007642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884378-fe94-4ca1-b011-93bd53257f68 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:41.059627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:41.107765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:41.108407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:41.108682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:41.108953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:41.109599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:41.111419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:12:41.111419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:12:41.111419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:12:41.112594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fce51-cf64-43a0-aad7-70164ff2627e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:41.132064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 9 11:12:40 2022] GET /identity/v3/projects => generated 2553 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:12:43.138124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:43.138642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:43.138642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:43.140582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1479] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:43.148824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b00b7e-a4ce-4607-8610-254eb50d1c6f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:43.149287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b00b7e-a4ce-4607-8610-254eb50d1c6f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:43.149388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b00b7e-a4ce-4607-8610-254eb50d1c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:43.219578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95b00b7e-a4ce-4607-8610-254eb50d1c6f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:43.220795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95b00b7e-a4ce-4607-8610-254eb50d1c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:43.463611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:12:43.507455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:43.510044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:43.524653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:43.681204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf80d4-8ab8-4fc3-8ed3-286830621610 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:43.769928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:43.804201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:43.804850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:43.805113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:43.805374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:43.806985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:43.904563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:43.906155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88b406-e78c-4279-b972-4fccd11cebdf service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:43.925166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:46.112515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:46.113410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:46.113410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:46.114918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1482] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:46.120838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0ead-9c53-40e0-bafa-3444372603df None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:46.121280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0ead-9c53-40e0-bafa-3444372603df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:46.121469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0ead-9c53-40e0-bafa-3444372603df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:46.193106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c47d0ead-9c53-40e0-bafa-3444372603df None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:46.194214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c47d0ead-9c53-40e0-bafa-3444372603df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:46.243274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:46.298595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:46.338449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:46.340085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:46.365496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:46.463191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd27ca-5c3a-4626-94f8-a33404e7a2d3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:46.550184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:46.550791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:46.551066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:46.551329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:46.552948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:46.630407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:46.668827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:46.670284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc471e55-9298-4616-896b-b8bf5fe3192a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:46.689413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:48.781722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:48.782727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:48.782727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:48.784817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1485] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:48.793653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d811e2-88ee-4a80-818a-105a3f972d8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:48.794099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d811e2-88ee-4a80-818a-105a3f972d8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:48.794202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d811e2-88ee-4a80-818a-105a3f972d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:48.868463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f9d811e2-88ee-4a80-818a-105a3f972d8e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:48.869698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9d811e2-88ee-4a80-818a-105a3f972d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:48.971971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:49.019256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:49.020832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:49.134241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb95c0-04ca-4b88-9414-380e717824c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:49.207128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:49.208964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:49.209562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:49.209834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:49.210105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:49.212186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:49.336436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:49.337872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2a950-65f7-4a11-bf17-8d4e9095d9bb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:49.357955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:51.546290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:51.547217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:51.547217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:51.549507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1488] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:51.559021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f06846-28f4-413b-8682-6855db14f38c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:51.559806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f06846-28f4-413b-8682-6855db14f38c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:51.560367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f06846-28f4-413b-8682-6855db14f38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:51.653114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-67f06846-28f4-413b-8682-6855db14f38c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:51.656200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67f06846-28f4-413b-8682-6855db14f38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:51.714355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:51.768618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:51.827003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:51.829491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:51.876682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:51.969525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87cdcc-fa00-4117-8631-5020f8562d61 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:52.044886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:52.045491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:52.045772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:52.046034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:52.047641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:52.137814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:52.170561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:52.172002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de0cf8-0ada-4ff8-a90c-9d2384895954 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:52.190822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:12:54.411304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:54.411908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:54.411908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:54.413201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1491] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 9 11:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:12:54.419787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ce66-9733-45e2-bdce-e9631714b715 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:54.420588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ce66-9733-45e2-bdce-e9631714b715 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:54.420765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877ce66-9733-45e2-bdce-e9631714b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:54.500707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8877ce66-9733-45e2-bdce-e9631714b715 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:12:54.501902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8877ce66-9733-45e2-bdce-e9631714b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:54.610031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 9 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:12:54.650152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:12:54.652367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:54.766700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc6b87-f7b0-4eb5-956d-b35a64a4086b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:12:54.844651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:12:54.845426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:12:54.845748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:12:54.846025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:12:54.847637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:12:54.861971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:12:54.974859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:12:54.976311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e2e78-6ba7-4c39-9bd5-e18de0c2592c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:12:54.996285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:13:04.607559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:13:04.993886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=50 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:12.170839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:12.171446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:12.171446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:12.247838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:14:12.249059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:12.356050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1494] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:14:12.364682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:12.365206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:12.365206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:12.446159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:14:12.447358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:12.551486 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1495] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:14:12.558044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:12.559275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:12.571496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69347a0b-c608-465c-8a3f-183215fbf542 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:12.639581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:12.672574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:12.673153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:12.673418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:12.673680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:12.674314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:14:12.674607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:12.675619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:12.675619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:14:12.675619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:14:12.676516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:12.687659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:14:12.699405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:12.708866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61c90-4e68-44f2-873e-1a7e926700c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:12.777697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:12.816925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:12.817525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:12.817795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:12.818079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:12.818716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:12.820573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:12.820573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:14:12.820573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:14:12.821945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:12.878477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:14:12 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:14:12.883392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:12.885022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:12.920730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6eeb8-68b8-43d5-9ee8-b51b44586d27 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:13.028788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:13.029405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.029642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.029938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.030560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:13.031594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:13.031594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:14:13.031594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:14:13.032815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:13.090143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:14:12 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:14:13.094539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:13.096120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:13.160862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768ec02-501f-4b3d-b829-7d8a66f2e1ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:13.245137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:13.256411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:13.257352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.257619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.257882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.258774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:13.259982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:13.259982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:14:13.259982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:14:13.260892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:13.273143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:14:13.277619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:13.279190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:13.290602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec726a-4698-405a-9f4b-af348964d612 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:13.316420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:13.392568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:13.393168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.393435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.393693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.394319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:13.395330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:13.395330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:14:13.395330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:14:13.396203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7e68e-e344-4462-b5db-65f13275614e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:13.409297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:14:13.413693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:13.415362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:13.450466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ba1ac-775e-40af-9d0a-f37dde7e44c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:13.553556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:13.554117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.554388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.554653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] PATH_INFO: `/v3/projects/8e15d2a2dc7e4c0691fab22f7e19af60/users/7ecc48702eac46ed89d40369f741a81c/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.576892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e15d2a2dc7e4c0691fab22f7e19af60, user_id=7ecc48702eac46ed89d40369f741a81c, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:13.577939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:13.577939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:14:13.577939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:14:13.579848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798753bd-e797-49c0-8f81-2a8cd2c8e7dd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:13.610692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:14:13 2022] PUT /identity/v3/projects/8e15d2a2dc7e4c0691fab22f7e19af60/users/7ecc48702eac46ed89d40369f741a81c/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:14:13.623922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de565e64-3337-438a-a214-66d69ffc7216 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.624201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de565e64-3337-438a-a214-66d69ffc7216 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.624500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de565e64-3337-438a-a214-66d69ffc7216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.685426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-de565e64-3337-438a-a214-66d69ffc7216 None None] MFA Rules not processed for user `7ecc48702eac46ed89d40369f741a81c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:14:13.686682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de565e64-3337-438a-a214-66d69ffc7216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:13.781567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1502] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:14:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:14:13.795472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.795776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.796044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.797427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1503] 10.222.0.27 () {54 vars in 964 bytes} [Tue Aug 9 11:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:14:13.802187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc38-2831-4fe5-a8f6-7f2780f26f48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:13.802464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc38-2831-4fe5-a8f6-7f2780f26f48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:13.802738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5dc38-2831-4fe5-a8f6-7f2780f26f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:13.836353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:13.887734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3b5dc38-2831-4fe5-a8f6-7f2780f26f48 None None] MFA Rules not processed for user `befb4776ff7b4014997d32e4697f29a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:14:13.889535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3b5dc38-2831-4fe5-a8f6-7f2780f26f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:13.974641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Tue Aug 9 11:14:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:14:13.982782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:13.984361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:14.040786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:14.104112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b65bf-c596-4791-aa34-312856f673f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:14.183871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:14.184509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:14.184784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:14.185051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:14.186654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:14.312676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:14.314123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:14.317605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:14.332899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:14:14.382419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057ddf-1d85-4d3f-9667-670e7968ad30 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:14.382881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057ddf-1d85-4d3f-9667-670e7968ad30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:14.382982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057ddf-1d85-4d3f-9667-670e7968ad30 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:14.384309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1506] 10.222.0.27 () {52 vars in 830 bytes} [Tue Aug 9 11:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:14:14.389494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:14.390716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:14.495573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d00f0-0601-4d4a-bd82-7e3589b6f972 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:14.599391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:14.600012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:14.600305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:14.600570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] PATH_INFO: `/v3/users/7ecc48702eac46ed89d40369f741a81c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:14.614079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecc48702eac46ed89d40369f741a81c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:14.615027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:14.615027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:14:14.615027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:14:14.616205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:14.629053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:14.694648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecc48702eac46ed89d40369f741a81c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:14.695071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecc48702eac46ed89d40369f741a81c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:14.706421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecc48702eac46ed89d40369f741a81c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:14.729109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecc48702eac46ed89d40369f741a81c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:14.729656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecc48702eac46ed89d40369f741a81c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:14.750716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:14:14 2022] DELETE /identity/v3/users/7ecc48702eac46ed89d40369f741a81c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:14:14.755335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:14.755612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:14.755876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:14.814016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:14:14.815207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:14.892787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:14.906670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1508] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:14:14.916895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:14.918136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:15.018521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b70151-2e7e-49e5-8606-cc5c64b0c1f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:15.074015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:15.103645 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:15.104259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:15.104563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:15.104829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:15.106438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:15.221784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:15.223121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af0638-792f-41d6-8071-d9d71aef28f6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:15.242553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:14:15.702516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:14:15.703866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:14:15.716814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7d127-000c-4e73-8921-bb93037608ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:14:15.817588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:14:15.818215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:14:15.818484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:14:15.818844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] PATH_INFO: `/v3/projects/8e15d2a2dc7e4c0691fab22f7e19af60` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:14:15.828843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e15d2a2dc7e4c0691fab22f7e19af60)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:14:15.829878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:14:15.829878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:14:15.829878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:14:15.831058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:14:15.896821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e15d2a2dc7e4c0691fab22f7e19af60', 'request_id': 'req-dc2d6c03-3898-48de-aade-8d469a66df72'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:15.903401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e15d2a2dc7e4c0691fab22f7e19af60', 'request_id': 'req-dc2d6c03-3898-48de-aade-8d469a66df72'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:15.915555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e15d2a2dc7e4c0691fab22f7e19af60', 'request_id': 'req-dc2d6c03-3898-48de-aade-8d469a66df72'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:15.958409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] The token cache is being invalidate because project 8e15d2a2dc7e4c0691fab22f7e19af60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:14:15.958838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:15.959696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc2d6c03-3898-48de-aade-8d469a66df72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:14:15.961548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:14:15 2022] DELETE /identity/v3/projects/8e15d2a2dc7e4c0691fab22f7e19af60 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:14:25.240057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=62 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:14:25.954593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:15:49.422928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:49.423835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:49.423835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:49.425775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:15:49.430733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487726-902a-41ae-bd6f-3be1449961f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:49.431191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487726-902a-41ae-bd6f-3be1449961f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:49.431293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06487726-902a-41ae-bd6f-3be1449961f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:49.530598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06487726-902a-41ae-bd6f-3be1449961f5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:15:49.531799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06487726-902a-41ae-bd6f-3be1449961f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:15:49.643344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:15:49.796582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:15:49.800446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:15:49.942519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70cf3e-92d2-4cf3-8929-15d2a23edc06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:15:49.950155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:15:49.950768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:49.951038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:49.951300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:49.959655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:15:49.962069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb5d44-6406-4ba9-8b3f-cbd19b0de074 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:15:49.971542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:15:50.816071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b507bb-a89b-4583-a639-5e08725182f6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:50.816967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b507bb-a89b-4583-a639-5e08725182f6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:50.816967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b507bb-a89b-4583-a639-5e08725182f6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:50.819209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:15:50.825027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d16c-5240-4c91-9024-30b8ab276376 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:50.825477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d16c-5240-4c91-9024-30b8ab276376 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:50.825584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d16c-5240-4c91-9024-30b8ab276376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:50.847866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4e54d16c-5240-4c91-9024-30b8ab276376 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:15:50.849103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e54d16c-5240-4c91-9024-30b8ab276376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:15:50.858867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:15:51.010030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b507bb-a89b-4583-a639-5e08725182f6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:15:51.024751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b507bb-a89b-4583-a639-5e08725182f6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:15:51.030107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:15:51.030707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:51.030979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:51.031245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:51.040014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:15:51.041655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174246f2-6d3f-4fd9-bedd-4a856b3c5dd9 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:15:51.051136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:15:51.563806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daa32-8ea8-45c7-ad66-6e7032758892 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:51.564041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daa32-8ea8-45c7-ad66-6e7032758892 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:51.564359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daa32-8ea8-45c7-ad66-6e7032758892 None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:51.565681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1517] 10.222.0.42 () {54 vars in 925 bytes} [Tue Aug 9 11:15:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:15:51.573566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519cbc-6a08-4b26-bddc-17f03abb0bcd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:51.573846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519cbc-6a08-4b26-bddc-17f03abb0bcd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:51.574121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41519cbc-6a08-4b26-bddc-17f03abb0bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:51.640574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41519cbc-6a08-4b26-bddc-17f03abb0bcd None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:15:51.641785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41519cbc-6a08-4b26-bddc-17f03abb0bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:15:51.659966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:15:51.726649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Aug 9 11:15:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:15:55.566813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3727b3-384a-4c00-b888-81b4850ea583 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:55.567428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3727b3-384a-4c00-b888-81b4850ea583 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:55.567428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3727b3-384a-4c00-b888-81b4850ea583 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:55.568754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1519] 10.222.0.27 () {54 vars in 961 bytes} [Tue Aug 9 11:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:15:55.575481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6e19e-4f68-4b31-8b2a-65b71718cad0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:55.576156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6e19e-4f68-4b31-8b2a-65b71718cad0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:55.576493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6e19e-4f68-4b31-8b2a-65b71718cad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:55.601880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0ba6e19e-4f68-4b31-8b2a-65b71718cad0 None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:15:55.603083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ba6e19e-4f68-4b31-8b2a-65b71718cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:15:55.624461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Aug 9 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:15:55.636724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3727b3-384a-4c00-b888-81b4850ea583 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:15:55.652965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3727b3-384a-4c00-b888-81b4850ea583 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:15:55.658559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:15:55.659146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:15:55.659417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:15:55.659682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:15:55.668450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:15:55.670024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a23ac5-6a98-476f-9cf6-75a5bed0172e service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:15:55.680159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:02.283545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415052-5419-4d15-a11e-3d3620e81ef7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:02.284492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415052-5419-4d15-a11e-3d3620e81ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:02.284492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415052-5419-4d15-a11e-3d3620e81ef7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:02.286650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:16:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:02.292311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5020c-0b26-40c3-bca0-1ff223e18f6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:02.293027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5020c-0b26-40c3-bca0-1ff223e18f6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:02.293201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5020c-0b26-40c3-bca0-1ff223e18f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:02.320680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3c5020c-0b26-40c3-bca0-1ff223e18f6d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:02.321887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3c5020c-0b26-40c3-bca0-1ff223e18f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:02.331434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:02.485601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55415052-5419-4d15-a11e-3d3620e81ef7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:02.507812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55415052-5419-4d15-a11e-3d3620e81ef7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:02.514410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:02.515001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:02.515271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:02.515533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:02.523532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:02.525165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939782a-02f4-4280-bd0a-e39d7440c07a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:02.534555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:03.346847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944697b-9437-4174-b782-c59d69df108a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:03.347451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944697b-9437-4174-b782-c59d69df108a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:03.347451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944697b-9437-4174-b782-c59d69df108a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:03.348756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 11:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:03.354559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccf61-1be0-45a6-9486-28fd27cff20e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:03.354995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccf61-1be0-45a6-9486-28fd27cff20e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:03.355103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccf61-1be0-45a6-9486-28fd27cff20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:03.377537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a32ccf61-1be0-45a6-9486-28fd27cff20e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:03.378731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a32ccf61-1be0-45a6-9486-28fd27cff20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:03.389407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 11:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:03.543958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944697b-9437-4174-b782-c59d69df108a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:03.566212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944697b-9437-4174-b782-c59d69df108a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:03.572658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:03.573671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:03.573945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:03.574209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:03.582539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:03.584212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a3454-ef1b-4e1d-b5bf-46cc33f84edb service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:03.594340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:07.662226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:07.663143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:07.663143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:07.665224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:07.672016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9b48b-d502-49da-91a2-617abe1b0d8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:07.672525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9b48b-d502-49da-91a2-617abe1b0d8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:07.672525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9b48b-d502-49da-91a2-617abe1b0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:07.693595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b8f9b48b-d502-49da-91a2-617abe1b0d8a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:07.694797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8f9b48b-d502-49da-91a2-617abe1b0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:07.703894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:07.782066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:07.784128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:07.874280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60966934-759f-430a-8fc6-26f8c7a52d81 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:07.880903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:07.881511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:07.881782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:07.882044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:07.892636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:07.894081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e98fc-6bbe-4cc7-9dca-61bb29c9e78b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:07.903925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:09.872995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:09.873620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:09.873620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:09.874903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:09.880683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e3a3b-c800-4213-b678-5947ccaceffe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:09.881106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e3a3b-c800-4213-b678-5947ccaceffe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:09.881225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e3a3b-c800-4213-b678-5947ccaceffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:09.908552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a57e3a3b-c800-4213-b678-5947ccaceffe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:09.909745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a57e3a3b-c800-4213-b678-5947ccaceffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:09.920617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:09.957716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:09.959796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:09.974442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc6701-9338-4e54-ab1c-c3fcd7f8488b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:09.978276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:09.978876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:09.979143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:09.979411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:09.989159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:09.990619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e359e07-e484-4422-bff2-34e845068ea5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:10.000472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:13.619922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:13.620584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:13.620584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:13.621839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:13.628190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c5b9-4852-4d8a-bec2-469d5d50c1b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:13.628693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c5b9-4852-4d8a-bec2-469d5d50c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:13.628790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c5b9-4852-4d8a-bec2-469d5d50c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:13.653647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b238c5b9-4852-4d8a-bec2-469d5d50c1b9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:13.654846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b238c5b9-4852-4d8a-bec2-469d5d50c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:13.663838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:13.970431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:13.972802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:14.050348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:16:14.068419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85e4e3-80f2-433f-87f7-b986368dd6ac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:14.072256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:14.072929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:14.073202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:14.073465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:14.082688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:14.084126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f50f9-f465-4845-85b8-e48354c0c419 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:14.094381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:16.095799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:16.096387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:16.096387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:16.097707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 11:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:16:16.103363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3238c-1cd9-4e40-b027-5293a3000196 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:16.103852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3238c-1cd9-4e40-b027-5293a3000196 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:16.103852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3238c-1cd9-4e40-b027-5293a3000196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:16.123890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e0e3238c-1cd9-4e40-b027-5293a3000196 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:16:16.125102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0e3238c-1cd9-4e40-b027-5293a3000196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:16.133213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:16:16.432818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:16:16.434975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:16:16.443143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:16:16.449908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb2b62-2630-4e06-8eab-20590fdcb046 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:16:16.453952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:16:16.454547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:16:16.454821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:16:16.455086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:16:16.464759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:16:16.466590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133223f-df8e-4639-8eec-7b816fee9587 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:16:16.476220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:16:26.119330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:16:26.472152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:41.393649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.396340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.396340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.431440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:41.433225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:41.448726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1540] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:17:41.458718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.459782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.460285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.488060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:41.492324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:41.501363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1541] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:41.506903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.523085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fcef3-db12-4787-a751-fc46f790a132 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.532167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.532831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.533108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.533377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.534039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:41.534337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.535957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.556197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:41.564737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.578224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87d497-ddc0-4307-9572-af62424dacaa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.587610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.588713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.589210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.592879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.593964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.596537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.656801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1543] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:41.660658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.675519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb1350-330c-4c8c-bdcc-231b54228a9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.749287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.749920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.750196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.750470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.751129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.753518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.792260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1544] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:17:41.796901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.813004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da594a2-2bf5-4f0d-9fe9-d400241860cd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.821320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.822398 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.822921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.823496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.824570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.826445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.844656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1545] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:41.848613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.864250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f30d33-1cd1-48ee-b8a7-728b135b430d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.873020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.873774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.874056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.874325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.874979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.876646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98852506-a713-4c71-9a5c-ede2c2cb3c3b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.893954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1546] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:41.898243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:41.905791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa969632-0ddb-4018-a976-c1cc0d5768cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.906135 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa969632-0ddb-4018-a976-c1cc0d5768cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.906312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa969632-0ddb-4018-a976-c1cc0d5768cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.913513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796829ad-b3ac-4f8c-afaf-5922821e5946 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:41.931698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:41.932431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.932709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.933131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] PATH_INFO: `/v3/projects/0487db6164f74d63ae08bfc5607ee4cc/users/1a96034d894d4fe39777baee7dcc4f95/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.934571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa969632-0ddb-4018-a976-c1cc0d5768cf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:41.935804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa969632-0ddb-4018-a976-c1cc0d5768cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:41.952458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1547] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:17:41.953742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b93-1443-459b-90a1-f575e3ec0a92 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:41.954023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b93-1443-459b-90a1-f575e3ec0a92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:41.954291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b93-1443-459b-90a1-f575e3ec0a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:41.970425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0487db6164f74d63ae08bfc5607ee4cc, user_id=1a96034d894d4fe39777baee7dcc4f95, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:41.973223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f67219-97e5-43e8-87f8-aeb970ece2fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:41.987195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b917b93-1443-459b-90a1-f575e3ec0a92 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:41.988479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b917b93-1443-459b-90a1-f575e3ec0a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.008418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1548] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.009702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517b0e-b1ee-4ca8-aa48-fa84df150a06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.009982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517b0e-b1ee-4ca8-aa48-fa84df150a06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.010251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3517b0e-b1ee-4ca8-aa48-fa84df150a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.011774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1549] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:41 2022] PUT /identity/v3/projects/0487db6164f74d63ae08bfc5607ee4cc/users/1a96034d894d4fe39777baee7dcc4f95/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:17:42.013214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc6e8-38c1-4da3-89c1-8ae2d936ccf6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.013501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc6e8-38c1-4da3-89c1-8ae2d936ccf6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.013763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dc6e8-38c1-4da3-89c1-8ae2d936ccf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.040536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e3517b0e-b1ee-4ca8-aa48-fa84df150a06 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.041751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3517b0e-b1ee-4ca8-aa48-fa84df150a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.053464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-497dc6e8-38c1-4da3-89c1-8ae2d936ccf6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.054880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-497dc6e8-38c1-4da3-89c1-8ae2d936ccf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.137494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1550] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.138220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1551] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.138932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2b19-ffed-48bc-a490-0b69257c5074 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.139285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2b19-ffed-48bc-a490-0b69257c5074 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.139560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2b19-ffed-48bc-a490-0b69257c5074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.140553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.141036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.142088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.161653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-afda2b19-ffed-48bc-a490-0b69257c5074 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.162946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afda2b19-ffed-48bc-a490-0b69257c5074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.171049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1552] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.172174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382fe76-eed3-43aa-9681-65989724fcc0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.172572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382fe76-eed3-43aa-9681-65989724fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.172841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382fe76-eed3-43aa-9681-65989724fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.177958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] MFA Rules not processed for user `1a96034d894d4fe39777baee7dcc4f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.179187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.192185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1382fe76-eed3-43aa-9681-65989724fcc0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.193433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1382fe76-eed3-43aa-9681-65989724fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.201261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1553] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.202849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95193-a753-4ebe-b214-789500b2eaae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.202849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95193-a753-4ebe-b214-789500b2eaae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.203058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95193-a753-4ebe-b214-789500b2eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.221971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59d95193-a753-4ebe-b214-789500b2eaae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.223150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59d95193-a753-4ebe-b214-789500b2eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.232035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1554] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.233341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34500b2-a1f6-4390-9f22-9e349cba158a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.233616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34500b2-a1f6-4390-9f22-9e349cba158a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.233886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34500b2-a1f6-4390-9f22-9e349cba158a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.244869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1555] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.245778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.253663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c34500b2-a1f6-4390-9f22-9e349cba158a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.254826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c34500b2-a1f6-4390-9f22-9e349cba158a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.260817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed24fdc-bf83-4819-b717-14a7c07a9961 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.263233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1556] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.265660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.266494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.266924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.266924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.267663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.267962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.268069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.268396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.268668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.269076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:17:42.269076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:17:42.269076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:17:42.270046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e46a-c684-42b4-ab59-957722c13c01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.283041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1557] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.284500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d5f2-a8f4-4d6e-9193-2e3093026c24 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.284788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d5f2-a8f4-4d6e-9193-2e3093026c24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.285058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0d5f2-a8f4-4d6e-9193-2e3093026c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.292530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.293977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.294205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:42.303686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1558] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.304528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.310675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6ef0d5f2-a8f4-4d6e-9193-2e3093026c24 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.313059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ef0d5f2-a8f4-4d6e-9193-2e3093026c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.314579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746bfb4-f3dd-45ae-b2b6-57b8b034ef62 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.319705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.320331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.320609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.320878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.321159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1559] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.321550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.321851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.322322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.322593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.322854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.323449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431959f-c626-49c1-9a77-9862c71cd1ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.333054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:42.340441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1560] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.341961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e561f4-5633-4251-878f-1b398c7c7dbe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.342199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.342600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e561f4-5633-4251-878f-1b398c7c7dbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.342878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e561f4-5633-4251-878f-1b398c7c7dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.343343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.351333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1561] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.355049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.362594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-75e561f4-5633-4251-878f-1b398c7c7dbe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.363753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d0c37-aa43-40d6-8100-66327d9fddc1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.363868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75e561f4-5633-4251-878f-1b398c7c7dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.368783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.369373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.369644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.369908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.370545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.370837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.371758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1562] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.372487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528deb-c696-4271-bc9b-87bcad5d2dab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.372957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648502-3bc7-4415-90a7-314660db1652 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.373238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648502-3bc7-4415-90a7-314660db1652 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.373505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53648502-3bc7-4415-90a7-314660db1652 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.374717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1563] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 9 11:17:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:17:42.375417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648502-3bc7-4415-90a7-314660db1652 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.387155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1564] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.388421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391d708-5ceb-4514-8d68-91c49da27221 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.388801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391d708-5ceb-4514-8d68-91c49da27221 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.389009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391d708-5ceb-4514-8d68-91c49da27221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.390026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53648502-3bc7-4415-90a7-314660db1652 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.394560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.395176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.395419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.395685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.396390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.397388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:17:42.397388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:17:42.397388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:17:42.399265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.408823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1391d708-5ceb-4514-8d68-91c49da27221 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.410019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1391d708-5ceb-4514-8d68-91c49da27221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.418075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1565] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.419187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.419462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.419729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.433432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1566] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.434161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.442706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.443994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.499090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bc06-21db-471c-a82a-c065006a3a7e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.504747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1567] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.505305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.505692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.505970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.506242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.506870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.508902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.509595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.518696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943af5-2b8f-4657-b819-52776d93f1d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.523636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.524226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.524525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.524791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.525450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.525729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.527321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9037a-a649-4ac2-95bf-9174cde00517 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.542894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1568] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.543113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.543232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.543337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.543337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.543337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.582910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:42.584186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:42.610931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a28848-fc5a-4af1-8c7a-15e3b64855c7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.615922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.616556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.616814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.617084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.617762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.618060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.619679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.632918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1570] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.633647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.643111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e3eac-a482-432d-b224-54add1d2fad6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.647845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.648449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.648705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.648966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.649568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.651426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.658330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1571] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:42.659178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.670112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8119f25-3d83-4dda-9a7a-b666d78b2338 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.675138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.675831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.676015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.676294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.676955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.677308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.678902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.680594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.681391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.693459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1573] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.694301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.705753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b94a4f-160a-4fc5-84c1-bc404a611334 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.713879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:42.727173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:42.746463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705b5c-a0a0-42fc-ba4c-a96c84d7edd1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.751024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.751662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.751934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.752334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.753300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.755253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.763245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.763757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.764039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.764582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.765009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:42.765313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.767784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.777691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1574] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.778630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.787589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1575] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:42.788256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.794440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c8604-e4ca-4ef3-a8bb-b426dd78044a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.798862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054525-b342-4025-aa23-104bfea047cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.799629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.800248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.800617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.800811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.801436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.803857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.804106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.804853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.805240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.805627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.806327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.808403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.826367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1576] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.828076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.837131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34650d-1612-4774-99e7-04205f14e08c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.838713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.839436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.897195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.897790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.898080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.898341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.898956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.900889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.912367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a9a0a-0371-421d-8b5b-31c8330f1f47 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:42.915939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:42.916599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:42.917037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:42.917306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:42.931940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1578] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:42.932793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:42.981555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:42.984242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:42.994535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1579] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:42.997341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd837-8293-4537-8b3b-52b5b164c002 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.001185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.003298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.003776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.004137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.004400 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.005201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.007811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.012235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9fab-dce1-4f16-b6eb-0512f105de5a service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.018699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.020159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.020521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.020793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.021488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.023548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.037720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:42 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.039279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.051814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b93783-4651-43f2-a425-f933b7fc3d80 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.054489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1581] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.055749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.066055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.072073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.132836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.133945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.134412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.134941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.136119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.137544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f25a05-bf5a-48ba-8b0a-e779984cf6bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.139058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.144633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.145564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.146089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.146540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.147560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.150637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.157241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1582] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.158657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.173869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0946e2-c949-4b6f-8790-a1511248abb0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.180485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.181303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.181734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.182119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.182848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.185565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.192047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1583] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.193674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.203577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1584] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.204783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.221044 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869e7d-28f7-4bd8-824a-60150b378a00 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.281886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeff27e-c2df-410a-85b9-e76a581bbed0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.286827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.287487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.287760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.288026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.288672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.289679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:17:43.289679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:17:43.289679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:17:43.290864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.292703 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.293308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.293581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.293845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.294459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.296398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.315832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.315832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.327582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1586] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.327582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.334089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e8a5-9c62-46c0-9a2a-2f8d9b879c1f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.337089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dabe46-ad91-44e8-86d0-95409f4371a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.341437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.341759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.342061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.342330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.342430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.342593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.342700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.342877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.343285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.343500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.344867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.345442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.355522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.356116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.366582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b563-abfe-41ad-8b29-1ae08c22bee9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.373378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1588] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.373378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.373642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.373642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.373642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.373642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.373982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.374689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.382918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d45aa-5e49-4bfd-8275-01d8370d83b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.387507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.388040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.388386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.388684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.388942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.389224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.389823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.391761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c889866-5650-458f-9980-5ee1dbeb0391 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.400983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e2578-241d-41ea-8d48-6b2a375b2b43 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.405940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.406509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.406782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.407083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.407678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.409270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.414814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1590] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:43.416140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b324-562d-4fca-9d32-6287c8296222 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.416465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b324-562d-4fca-9d32-6287c8296222 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.416731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b324-562d-4fca-9d32-6287c8296222 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.418309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1591] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 9 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:17:43.419043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691b324-562d-4fca-9d32-6287c8296222 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.425200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.425200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.433086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f28f7-4d66-410e-a46b-4d7807f9f519 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.435008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691b324-562d-4fca-9d32-6287c8296222 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.437919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.438585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.438927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.439182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.439672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.439787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.440248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.440533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.440796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.441432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.441954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.443151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.451682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.452502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.455846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.455846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.465460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f696c2-6057-4316-a87a-0e7ab1be7f24 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.465728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eabcd-23d5-4d0a-9a0c-ea3f05cf737e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.470130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.470722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.470872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.470980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.471279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.471399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.471604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.471895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.472002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.472569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.473498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.474172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.484067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.484470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.485031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.485599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.495788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1bfd6d-58d0-43f7-bdd0-3d55a6e550d3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.496217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b0340-12e3-4d72-b7fe-f1ae463a5968 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.500807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.501276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.501521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.501800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.501904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.502070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] PATH_INFO: `/v3/projects/1c7aaa02e0a2496cab5733c70c07c55d/users/3c4332d40eb641eda59f052470cb692c/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.502199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.502810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.503503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.505170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53412a9-2dcc-4184-a14c-d647c78252a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.516490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.518035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.518423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.518675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.518937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7aaa02e0a2496cab5733c70c07c55d, user_id=3c4332d40eb641eda59f052470cb692c, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.521909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.536415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1598] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:43 2022] PUT /identity/v3/projects/1c7aaa02e0a2496cab5733c70c07c55d/users/3c4332d40eb641eda59f052470cb692c/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:43.538978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:43.540170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:43.541108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.551130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.558214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.606451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab50671-7a5a-4249-8a3b-c8b71c3d8a54 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.611411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1599] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:43.612374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.617100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.617876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.618132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.619585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] PATH_INFO: `/v3/projects/8459b14a54824536b415b2b7336389df/users/69c8a0130c8e42c5b4a6ffe43d15865e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.623480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fd7cb-764c-406c-8149-7ee9f6392723 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.628042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.628685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.628956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.629219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.629850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.631476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.635442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8459b14a54824536b415b2b7336389df, user_id=69c8a0130c8e42c5b4a6ffe43d15865e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.638167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.641620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1600] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.645028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.654751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececc6f9-82f8-482c-82f4-9b2d6732bbcf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.655842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1601] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:43 2022] PUT /identity/v3/projects/8459b14a54824536b415b2b7336389df/users/69c8a0130c8e42c5b4a6ffe43d15865e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:43.656795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.716861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19a67c-a736-4187-a24a-6b027168d714 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.717857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.718486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.718788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.719062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.719736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.721448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.721580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.722100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.722377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.722655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] PATH_INFO: `/v3/projects/f938fbc477f34eedb17c9ead3084a207/users/2e6543b377f14949a6919af4e4e187b7/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.734743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.734743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.738452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f938fbc477f34eedb17c9ead3084a207, user_id=2e6543b377f14949a6919af4e4e187b7, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.741897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.742637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad14fb-9443-47a8-8f1f-8329492c492e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.747988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.748649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.748898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.749154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.749848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.751422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c68be-d0a1-46b0-be9d-7d8f028d400c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.757943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:43 2022] PUT /identity/v3/projects/f938fbc477f34eedb17c9ead3084a207/users/2e6543b377f14949a6919af4e4e187b7/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:43.758870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.764096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1604] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.765254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.765528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.765792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.793688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] MFA Rules not processed for user `3c4332d40eb641eda59f052470cb692c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:43.794813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:43.820371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce366238-91c7-4075-9c42-c0941468d6a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.824837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.825445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.825716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.825976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.826608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.828170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.838639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1605] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:43.839697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.849930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.861999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1606] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:43.863010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.873737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4174e0c-a73d-4a75-b021-9a35237013fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.879253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.880057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.880511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.880762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] PATH_INFO: `/v3/projects/7443ad1e03504e999842e1bf42fdf454/users/2a9775cefaf24e6c805069fddf2ff369/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.894978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7443ad1e03504e999842e1bf42fdf454, user_id=2a9775cefaf24e6c805069fddf2ff369, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.896056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:17:43.896056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:17:43.896056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:17:43.898047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2f659-e0ff-477f-aa84-efa4fe28d837 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.900122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428bc5d-2995-4aa0-bb76-4b39f66635d2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:43.900449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:43.903782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:43.904422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.904760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.904971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.913762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1607] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:43 2022] PUT /identity/v3/projects/7443ad1e03504e999842e1bf42fdf454/users/2a9775cefaf24e6c805069fddf2ff369/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:43.915370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa659-91b0-447d-9246-b83b7c846c79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:43.916509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa659-91b0-447d-9246-b83b7c846c79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:43.916792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daa659-91b0-447d-9246-b83b7c846c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:43.938794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7daa659-91b0-447d-9246-b83b7c846c79 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:43.940121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7daa659-91b0-447d-9246-b83b7c846c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:43.974360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:43.976042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:43.986739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1608] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:43.987595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:43.997586 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dead2b-876a-4d7a-b566-3c2b4ad668d6 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.003468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.007253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1609] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.007419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.007419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.007419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] PATH_INFO: `/v3/projects/cba1dd0b689b4fff847b6fd3787b143c/users/f4da6dd860bb45d1a703a18c2512ffe4/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.007700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.007700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.007700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.025945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba1dd0b689b4fff847b6fd3787b143c, user_id=f4da6dd860bb45d1a703a18c2512ffe4, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.026448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:44.027804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:44.028883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.035732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1610] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.036679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.047369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336a57a-651e-4a95-931b-884088bdf2a1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.058047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1611] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:43 2022] PUT /identity/v3/projects/cba1dd0b689b4fff847b6fd3787b143c/users/f4da6dd860bb45d1a703a18c2512ffe4/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:44.058047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.105471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.107125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.107125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.107125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] PATH_INFO: `/v3/projects/33404d64cb6a470fa3d1ad5ff116e7eb/users/e7926cb4ddc5499194fc986c040a17d7/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.122443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33404d64cb6a470fa3d1ad5ff116e7eb, user_id=e7926cb4ddc5499194fc986c040a17d7, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.125297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.125608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4d037-dcb6-4df8-ab7b-a11189a65852 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.130858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.131746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.131988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.132335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] PATH_INFO: `/v3/projects/06dd1efa43b043cf84f9808305d97bf0/users/601782d5743c47f380358021be28cc91/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.143033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1612] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:44 2022] PUT /identity/v3/projects/33404d64cb6a470fa3d1ad5ff116e7eb/users/e7926cb4ddc5499194fc986c040a17d7/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:44.143972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.155900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=06dd1efa43b043cf84f9808305d97bf0, user_id=601782d5743c47f380358021be28cc91, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.158899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f11904-599f-4f9a-a687-8b4e60263bcc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.175690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1613] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:44 2022] PUT /identity/v3/projects/06dd1efa43b043cf84f9808305d97bf0/users/601782d5743c47f380358021be28cc91/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:44.177449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.177965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.178458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.199694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:44.213267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] MFA Rules not processed for user `2a9775cefaf24e6c805069fddf2ff369`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:44.215327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:44.216408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3bb9d-9e62-4221-bdfc-cf27812f0631 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.240109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:44.275129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.275889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.276122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.276420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.296170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1614] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.296170 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.336952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.338434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.347256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:44.348193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.356247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c89ff-ada8-4896-864d-1aaddc3452d5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.357108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f19d9-a23d-41dc-b63e-b4987dd0464d service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.360036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.360787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.360992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.361145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.361364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.361576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.361907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.362096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.433519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.435395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7a5af-96ff-4c75-964b-2e70097a7ce2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.437036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.438586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dbb82-8cb6-4a4e-ac19-a15a7536f23e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.447560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:44.449113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cb8e0-ced1-4090-a2cd-1790001b86b7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.449456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cb8e0-ced1-4090-a2cd-1790001b86b7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.449788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cb8e0-ced1-4090-a2cd-1790001b86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.463280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:44.466935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.467223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.467492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.471704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a42cb8e0-ced1-4090-a2cd-1790001b86b7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:44.473188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a42cb8e0-ced1-4090-a2cd-1790001b86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:44.485831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1618] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.486987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.487356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.487717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.510036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:44.511569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:44.522203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1619] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.523097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.534389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d350d-e026-4fce-8b32-5ac6c2d47ea2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.545370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] MFA Rules not processed for user `e7926cb4ddc5499194fc986c040a17d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:44.545983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.546668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.547067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.547448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.548129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:44.548248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.550717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.589294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1620] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:44.590507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.610913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:44.674557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1621] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:44.676165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.683836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:44.689156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193fe9d-d7a4-46db-b095-e89ede2558ec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.753966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f572911-456e-4a6b-b830-01f0dc0156ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.755699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.756796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.756986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.757399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.764828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.767941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.768224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.770221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.887423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.888974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.914993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:44.916361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:44.950597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552c7b-128b-4d8f-bbd5-04ef8eb9ac70 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:44.961629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.968915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:44.969881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:44.970519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:44.970795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:44.971063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:44.971699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:44.977236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.014448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1623] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:45.016721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.020886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1624] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:44 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:45.033322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.045873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c41dd-ffa4-4062-a0e0-4a7852ffdedd service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.046727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1f77d-2dde-4610-91c3-f507f07a2dcb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.054940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.055558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.055833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.056104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.057348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.058130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.058543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.060535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.061305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.063983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73725fba-82eb-4818-99a3-2117d94851ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.064804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.066459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.090641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:17:45.091691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.118139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89559a-9875-4bff-85a1-1e0494936af6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.123980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.124623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.124902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.125170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] PATH_INFO: `/v3/projects/ce088013ad2d4146b5b43e66e0db538c/users/2e32546bc48d4d9e9b936f995cae00ee/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.130203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1626] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:45.133104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10233550-0877-42df-991e-f886999fafa2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.133414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10233550-0877-42df-991e-f886999fafa2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.134020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10233550-0877-42df-991e-f886999fafa2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.135540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1627] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 9 11:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:17:45.138816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10233550-0877-42df-991e-f886999fafa2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.161709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce088013ad2d4146b5b43e66e0db538c, user_id=2e32546bc48d4d9e9b936f995cae00ee, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.164524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dbe0a-53b6-40cb-ae4e-b4fcd2c9902b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.210989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1628] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:45 2022] PUT /identity/v3/projects/ce088013ad2d4146b5b43e66e0db538c/users/2e32546bc48d4d9e9b936f995cae00ee/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:45.214405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.215084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.215749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.243967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10233550-0877-42df-991e-f886999fafa2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.246181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:45.247605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:45.317448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:45.336579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:45.340577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1629] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:45.341989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.354757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.356388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.356997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.357586 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.358485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.360840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196524e-0a0c-4fc8-966a-37d173aacd44 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.394807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1630] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/users => generated 296 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:45.396318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956de6-f8d0-48a8-a392-f371be977597 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.396747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956de6-f8d0-48a8-a392-f371be977597 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.397154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956de6-f8d0-48a8-a392-f371be977597 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.400387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1631] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 9 11:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:17:45.402723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956de6-f8d0-48a8-a392-f371be977597 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.420764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956de6-f8d0-48a8-a392-f371be977597 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.427869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8995684-5bb6-4c9c-b9dc-079ef1cf8243 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.428112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.428854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.429295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.429758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.431163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.435908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.436093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.436820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.437495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.438057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.438834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.441085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5a6c4-ca93-499c-8f12-c84d4358a49c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.455861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1632] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:45.457957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.458242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.458551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.481266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1633] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:45.482399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.486892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:45.489042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:45.506272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1634] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:45.507561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.508138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cd080-0813-472d-a955-491782c2ec0a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.518354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.519558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.520078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.520818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] PATH_INFO: `/v3/projects/ce088013ad2d4146b5b43e66e0db538c/users/2e32546bc48d4d9e9b936f995cae00ee/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.525558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce088013ad2d4146b5b43e66e0db538c, user_id=2e32546bc48d4d9e9b936f995cae00ee, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.529033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dffd1-c865-4809-8ba1-6a8293c3dea3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.529869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.543476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.544324 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.544604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.544877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] PATH_INFO: `/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c/users/2ac365959d7044faa3e0a09e4294ab20/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.547489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1635] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:45 2022] PUT /identity/v3/projects/ce088013ad2d4146b5b43e66e0db538c/users/2e32546bc48d4d9e9b936f995cae00ee/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:45.558850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.574843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b3bf2454e142fcab203d9ba33ed54c, user_id=2ac365959d7044faa3e0a09e4294ab20, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.578146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765bcbd-2636-4966-bbb7-53ed4f8f8383 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.603686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1636] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:45 2022] PUT /identity/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c/users/2ac365959d7044faa3e0a09e4294ab20/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:45.606250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.606570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.607012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.646190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] MFA Rules not processed for user `2e32546bc48d4d9e9b936f995cae00ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:45.648036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:45.696977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8062e-b823-433e-8682-60fdc0a0299e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.764074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1637] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:45.765739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.808110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.809400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.809896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.810509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.829460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:45.851929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3d544-804b-4838-b930-0a8596738cfe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.857905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.859073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.859334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.859598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.860780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.862825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.879912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1638] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:45.880936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.892773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e9ea-86c8-4db8-a228-4b192775f9cf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.898235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.898898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.899170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.899434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.921617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.922501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586cf56-8716-4313-b73d-3afb36a7329d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.925189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.926913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.938451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:45.939316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:45.950714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9a22d-174b-42d5-b701-c65dd140a29b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:45.960458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:45.961125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.961391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.961654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] PATH_INFO: `/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c/users/2ac365959d7044faa3e0a09e4294ab20/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:45.962719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:45.964120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b3bf2454e142fcab203d9ba33ed54c, user_id=2ac365959d7044faa3e0a09e4294ab20, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:45.966732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:45.980157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1641] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:45 2022] PUT /identity/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c/users/2ac365959d7044faa3e0a09e4294ab20/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:45.990349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:45.990762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:45.991240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:46.040483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:46.065775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] MFA Rules not processed for user `2ac365959d7044faa3e0a09e4294ab20`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:46.067199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:46.174095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:46.258068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1642] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:46.315413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499ac7-48d0-4b55-8892-552478e4eff4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:46.321072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:46.321989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:46.322354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:46.322785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:46.477225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:46.483465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:46.489446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:46.495937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eaa8a-42fb-4968-b2cb-85fd90448270 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:46.503584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:46.504964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:46.505460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:46.505823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:46.527275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:46.549794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:46.551506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:46.571293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:46.815104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:47.002895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8ad39-2155-4edd-a594-f15c365a2e25 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:47.014234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:47.015013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:47.015431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:47.015782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] PATH_INFO: `/v3/users/1a96034d894d4fe39777baee7dcc4f95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:47.017209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a96034d894d4fe39777baee7dcc4f95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:47.018593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:17:47.018593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Aug 09 11:17:47.018593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: Aug 09 11:17:47.019982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:47.060185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:47.154852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:47.169859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff847f32-5d89-4c96-a8c0-0b97077e690d service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:47.180290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:47.181067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:47.181458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:47.181816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:47.189788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a96034d894d4fe39777baee7dcc4f95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:47.212257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a96034d894d4fe39777baee7dcc4f95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:47.229875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a96034d894d4fe39777baee7dcc4f95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:47.230591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a96034d894d4fe39777baee7dcc4f95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:47.276570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec96fde5-9cb0-4cfe-a307-5001668c3ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a96034d894d4fe39777baee7dcc4f95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:47.296405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:47.302836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:47.308423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1645] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:17:46 2022] DELETE /identity/v3/users/1a96034d894d4fe39777baee7dcc4f95 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:47.312825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:47.313181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:47.314772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:47.327929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:47.372228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:47.374030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:47.492628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1647] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:47.514760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:47.626015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2279ac8-e63f-4b1f-b5a1-0cf1d42d3eef service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:47.716947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:47.759735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:47.760401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:47.761003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:47.761003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:47.771661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:47.773808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a6e38-c578-4d25-aae0-9cddab0bce02 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:47.785884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:49.421587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:49.423410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:49.455518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6551af-1df2-49ef-82c7-52d6387b6f2e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:49.468043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:49.468694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:49.469639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:49.470070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] PATH_INFO: `/v3/projects/0487db6164f74d63ae08bfc5607ee4cc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:49.472647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0487db6164f74d63ae08bfc5607ee4cc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:49.475072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:49.628758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0487db6164f74d63ae08bfc5607ee4cc', 'request_id': 'req-dd58f07e-fb4c-4511-ad9a-de962077b5bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:49.646090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0487db6164f74d63ae08bfc5607ee4cc', 'request_id': 'req-dd58f07e-fb4c-4511-ad9a-de962077b5bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:49.665495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0487db6164f74d63ae08bfc5607ee4cc', 'request_id': 'req-dd58f07e-fb4c-4511-ad9a-de962077b5bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:49.687118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] The token cache is being invalidate because project 0487db6164f74d63ae08bfc5607ee4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:17:49.687489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:49.688917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd58f07e-fb4c-4511-ad9a-de962077b5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:49.690328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1649] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:17:49 2022] DELETE /identity/v3/projects/0487db6164f74d63ae08bfc5607ee4cc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:49.695246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:49.695746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:49.695895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:49.754210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:49.761860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:49.947919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1650] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:49.955963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:49.956547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:49.957096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.021569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:50.027645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:50.060447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1651] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:50.063849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.081792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a608d1c-bd95-489b-bd95-c2f62d786c34 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.089892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.090770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.091181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.091608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.092425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:50.092855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.095444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.115368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1652] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:50.116073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.123202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:50.128919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e473f71-6f30-4a0c-95cf-450f8fb749d8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.142219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.142882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.143258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.143445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.144160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.149530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.210054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1653] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:50 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:17:50.217382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.237478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de536-8ca2-44fb-8312-29e20fd9615f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.439702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.440855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.441268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.441427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.445237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.447718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.507207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1654] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:50 2022] POST /identity/v3/users => generated 287 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:50.508779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.526431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b6ee3-b589-4f71-a04d-aa69dba3c1cd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.549680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.552933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.554166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.554440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.557932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.563261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.610033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1655] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:50.615369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.631148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1d3b5-804f-4edc-9c9f-11a20778187e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.646989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.649559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.651241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.651623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.652492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.657899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf142-dc05-4112-b6a4-b6ff77fd7351 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.698797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:50.713229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1656] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:50.716475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060a80-0635-449a-b85c-a7d0fc17780e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:50.726624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:50.734724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.736184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.736727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] PATH_INFO: `/v3/projects/6f5cee66e85840d6bb8b4190b5562a11/users/72cf0edb4c7342bda279c60280fbf4f3/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.758343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5cee66e85840d6bb8b4190b5562a11, user_id=72cf0edb4c7342bda279c60280fbf4f3, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:50.765922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:50.812394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:50.812949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:50.813631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:50.816681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1657] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:50 2022] PUT /identity/v3/projects/6f5cee66e85840d6bb8b4190b5562a11/users/72cf0edb4c7342bda279c60280fbf4f3/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:50.906004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] MFA Rules not processed for user `72cf0edb4c7342bda279c60280fbf4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:50.907776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:51.036893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:51.196458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1658] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:51.197309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:51.199185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:51.417607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de30f6-4147-486e-9ab8-a8297f10e525 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:51.433003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:51.433694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:51.433969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:51.434231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:51.458444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:51.463415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76827c-e8e6-479b-a8a4-5ee7a2179df3 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:51.496728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1659] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:52.152095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:52.341041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00563d57-43e4-4797-a0ea-edb10c07ff1a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:52.347764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:52.349472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:52.350969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:52.351487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] PATH_INFO: `/v3/users/72cf0edb4c7342bda279c60280fbf4f3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:52.359843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cf0edb4c7342bda279c60280fbf4f3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:52.372638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:52.448642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cf0edb4c7342bda279c60280fbf4f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:52.453129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cf0edb4c7342bda279c60280fbf4f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:52.486849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cf0edb4c7342bda279c60280fbf4f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:52.518743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cf0edb4c7342bda279c60280fbf4f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:52.522602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cf0edb4c7342bda279c60280fbf4f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:52.564688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1660] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:17:52 2022] DELETE /identity/v3/users/72cf0edb4c7342bda279c60280fbf4f3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:52.566235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367152a-81c7-4273-aced-0001a27ea48e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:52.569989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367152a-81c7-4273-aced-0001a27ea48e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:52.570686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367152a-81c7-4273-aced-0001a27ea48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:52.611509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4367152a-81c7-4273-aced-0001a27ea48e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:52.614347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4367152a-81c7-4273-aced-0001a27ea48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:52.825813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1661] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:52.879205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:52.880952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:53.092142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c392-b013-4711-a7a5-6b3e06a4f35a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:53.096958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:53.097834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:53.098104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:53.098411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:53.117048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:53.119038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:53.124702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:53.140794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1662] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:53.974479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:53.975478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:53.975946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:54.021577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] MFA Rules not processed for user `601782d5743c47f380358021be28cc91`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:54.023403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:54.104661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:54.208412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1663] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:54.225254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:54.228256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:54.412191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdbdbe-b668-40e3-b719-982f9e9f3dbb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:54.419786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:54.420250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:54.420624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:54.420961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:54.440110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:54.441791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d740bb-3ae4-4965-b13c-6848a2f6ee66 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:54.475765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1664] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:17:55.364969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:55.369499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:55.408030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10196c5a-8d7d-4d6b-a127-0da9f26b07d9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:55.440006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:55.443757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:55.444504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:55.446849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] PATH_INFO: `/v3/projects/6f5cee66e85840d6bb8b4190b5562a11` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:55.452134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5cee66e85840d6bb8b4190b5562a11)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:55.454600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:55.599565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5cee66e85840d6bb8b4190b5562a11', 'request_id': 'req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:55.617631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5cee66e85840d6bb8b4190b5562a11', 'request_id': 'req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:55.634129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5cee66e85840d6bb8b4190b5562a11', 'request_id': 'req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:55.668402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] The token cache is being invalidate because project 6f5cee66e85840d6bb8b4190b5562a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:17:55.671406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:55.675774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd715f36-e53e-4a1e-a4a8-5fe54d95306d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:17:55.680309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1665] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:17:55 2022] DELETE /identity/v3/projects/6f5cee66e85840d6bb8b4190b5562a11 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:17:55.684883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:55.685421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:55.685684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:55.725205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:55.726588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:55.875916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1666] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:55.878446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:55.878840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:55.879202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:55.933970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:55.935481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:55.959484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1667] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:55.960681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:55.978151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c0b16-6716-4d59-aad0-a278255d4302 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:55.996145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:55.997123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:55.997518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:55.998011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:55.998784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:17:55.999178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.003523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.040373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1668] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:17:56.041709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:56.055647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3196f-19a0-4f4b-ba90-562691e9db79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:56.071147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:56.072045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.072505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.072901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.073679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.075882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.158385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1669] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:56 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:56.159098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:56.171918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bb364-b99b-4b23-9bab-ca4f6db7baab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:56.262583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:56.263359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.264011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.264141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.266235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.269091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.315830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:56.328900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1670] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:56 2022] POST /identity/v3/users => generated 278 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:56.330187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:56.344111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd549a-51c4-408f-8cc2-c3b0df4b6f18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:56.356488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:56.357356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.357797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.359379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.362342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.364554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.383559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:56.387668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1671] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:56.397066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb06f-838f-4063-afaa-5444f04d01cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:56.408719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:56.409355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.409632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.409901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.410581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.412212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e560c6d-128a-43b7-8e29-55b61b1b7cf8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.440893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1672] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:56.441764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:56.468021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a7c8d-4aee-4819-b523-726139dbcf39 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:56.508654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:56.512087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.512558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.513728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] PATH_INFO: `/v3/projects/3e2a289f595c433a9448f0f73947a39b/users/df814b36a7c642aea0fd072cba28732a/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.593638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2a289f595c433a9448f0f73947a39b, user_id=df814b36a7c642aea0fd072cba28732a, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:56.597881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:56.656588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1673] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:56 2022] PUT /identity/v3/projects/3e2a289f595c433a9448f0f73947a39b/users/df814b36a7c642aea0fd072cba28732a/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:17:56.663870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:56.664386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:56.664544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:56.725437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] MFA Rules not processed for user `df814b36a7c642aea0fd072cba28732a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:56.726696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:56.882868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1674] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:56.884170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.031981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:57.093564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9702d-8de9-48d6-a633-c16e0cbca4a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.117129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.118304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.118738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.119166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.120964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.123127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.198834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1675] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:17:56 2022] POST /identity/v3/projects => generated 357 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:57.202608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.226234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ea8f-f13a-4520-a9a0-d8c762191ac9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.408324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.409370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.409804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.410089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.410729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.414342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.469915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1676] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:17:57 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:17:57.473636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.489638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b87cc-0337-4d35-87ff-18f55353e24f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.501629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.502470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.502745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.503010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.504014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.524107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.563816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.564398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1677] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:57.576597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3279-d108-4870-9810-ca81fb833311 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.593233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.593859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.594132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.594403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] PATH_INFO: `/v3/projects/69e556169d9a49fca0cbcd7a082d2def/users/3b872ef01ae345f8b541c8b0fcd724c3/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.653350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e556169d9a49fca0cbcd7a082d2def, user_id=3b872ef01ae345f8b541c8b0fcd724c3, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.656088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.703594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1678] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:57 2022] PUT /identity/v3/projects/69e556169d9a49fca0cbcd7a082d2def/users/3b872ef01ae345f8b541c8b0fcd724c3/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:17:57.708015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.726165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe12c2-d437-491c-a53b-1a8c1d709235 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.872019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.872940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.873295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.873896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.875999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.879710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97a671-8beb-4b6c-b926-35130ee5e2e5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.901234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1679] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:17:57.905477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:57.920184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7235-a032-42bc-af7e-c2744cd5ee2e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:57.935460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:57.936334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:57.939298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:57.939709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] PATH_INFO: `/v3/projects/69e556169d9a49fca0cbcd7a082d2def/users/3b872ef01ae345f8b541c8b0fcd724c3/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:57.945267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e556169d9a49fca0cbcd7a082d2def, user_id=3b872ef01ae345f8b541c8b0fcd724c3, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:57.954547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:57.992458 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1680] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:17:57 2022] PUT /identity/v3/projects/69e556169d9a49fca0cbcd7a082d2def/users/3b872ef01ae345f8b541c8b0fcd724c3/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:17:58.006727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:58.007017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:58.007288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:58.077003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] MFA Rules not processed for user `3b872ef01ae345f8b541c8b0fcd724c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:58.079251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:58.317275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1681] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:17:58.477296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:58.479385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:58.557137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:58.645738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ddc2-71fc-4ff0-ab4a-0c61f240ac4f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:58.656337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:58.657167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:58.658406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:58.659874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:58.681384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:58.683385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ac10-52a2-4c9c-bdf5-0c25a48d03e0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:58.737034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1682] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:59.168098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:17:59.169894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:59.207787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5eacd-cc67-4703-8aad-f4181f8c514b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:17:59.220351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:17:59.221795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:59.222080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:59.222347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:59.323352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:17:59.396959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:17:59.398870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:17:59.424090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1683] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:17:59.567333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360b81-9b64-47cd-a834-e97b599fa139 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:17:59.567797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360b81-9b64-47cd-a834-e97b599fa139 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:17:59.567953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32360b81-9b64-47cd-a834-e97b599fa139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:17:59.608788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-32360b81-9b64-47cd-a834-e97b599fa139 None None] MFA Rules not processed for user `601782d5743c47f380358021be28cc91`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:17:59.611088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32360b81-9b64-47cd-a834-e97b599fa139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:17:59.856322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1684] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:00.358546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:00.364398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:00.545940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ea013-dff6-4cbd-9b7f-cd1864567af5 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:00.579498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:00.581398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:00.581867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:00.582318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] PATH_INFO: `/v3/users/3c4332d40eb641eda59f052470cb692c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:00.585080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4332d40eb641eda59f052470cb692c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:00.589904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:00.708204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4332d40eb641eda59f052470cb692c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:00.742480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4332d40eb641eda59f052470cb692c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:00.774858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4332d40eb641eda59f052470cb692c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:00.780915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4332d40eb641eda59f052470cb692c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:00.822880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbb006f7-559b-4d24-bf18-fa6a65c48f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4332d40eb641eda59f052470cb692c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:00.845287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1685] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:00 2022] DELETE /identity/v3/users/3c4332d40eb641eda59f052470cb692c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:00.848905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:00.849384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:00.849461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:00.892587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:00.893874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.093964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1686] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:01.184582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249ae5-d5f2-4846-ba5a-ad176efd2b24 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.184917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249ae5-d5f2-4846-ba5a-ad176efd2b24 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.185253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249ae5-d5f2-4846-ba5a-ad176efd2b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.186450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:01.189451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.275198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-64249ae5-d5f2-4846-ba5a-ad176efd2b24 None None] MFA Rules not processed for user `69c8a0130c8e42c5b4a6ffe43d15865e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:01.278249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64249ae5-d5f2-4846-ba5a-ad176efd2b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.377753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4c879-1949-4a48-87d6-0d904e68e5e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:01.396659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:01.397568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.397985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.398392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.418891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:01.424657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:01.426608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957f5a3-1105-4d22-bdaf-593b86096351 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:01.450705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:01.478695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1688] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:01.493606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.493889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.494163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.540798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b5ec-2e6a-4a64-8cbb-16f47f87afaa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.541333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b5ec-2e6a-4a64-8cbb-16f47f87afaa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.541647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14b5ec-2e6a-4a64-8cbb-16f47f87afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.566082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] MFA Rules not processed for user `69c8a0130c8e42c5b4a6ffe43d15865e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:01.568453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.594390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1689] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:01.605545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:01.607420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.643291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7a14b5ec-2e6a-4a64-8cbb-16f47f87afaa None None] MFA Rules not processed for user `2e6543b377f14949a6919af4e4e187b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:01.644675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a14b5ec-2e6a-4a64-8cbb-16f47f87afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.665251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:01.831159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adeb70-27aa-4f41-8906-33f61d06f845 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:01.871592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1690] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:01.871756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:01.873052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.873523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.873927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.875499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4492c9-abed-4b09-a617-d031d6a33cd0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.876241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4492c9-abed-4b09-a617-d031d6a33cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.876658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4492c9-abed-4b09-a617-d031d6a33cd0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.887247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1691] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 9 11:18:01 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:01.889080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:01.889757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:01.889856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:01.901599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:01.906735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:01.931361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:01.967793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:01.970074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:01.994350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1693] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:02.004077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:02.011600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:02.040564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67894dd1-bab4-4e2a-aeba-bac428b95784 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:02.054896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:02.057096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:02.057382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:02.057708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:02.080990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:02.092541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124b18c-064b-4cdc-b3ea-2f6a45248d2d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:02.124155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1694] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:02.171695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:02.175431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:02.209615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a95d4-28f3-42fb-b0a1-5211635a49d8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:02.221903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:02.223483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:02.223483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:02.223483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] PATH_INFO: `/v3/users/e7926cb4ddc5499194fc986c040a17d7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:02.225098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7926cb4ddc5499194fc986c040a17d7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:02.231682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:02.357611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7926cb4ddc5499194fc986c040a17d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.367723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7926cb4ddc5499194fc986c040a17d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.380776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7926cb4ddc5499194fc986c040a17d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.381670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7926cb4ddc5499194fc986c040a17d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.397206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7926cb4ddc5499194fc986c040a17d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.404864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1695] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:02 2022] DELETE /identity/v3/users/e7926cb4ddc5499194fc986c040a17d7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:02.411461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:02.415514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:02.417333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:02.458114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:02.459916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:02.638836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1696] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:02.667638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:02.669196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:02.679495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:02.681182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:02.716897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8c4b-0a50-45bd-9ef1-cfeb9c268f72 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:02.736252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:02.737012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:02.737396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:02.737769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] PATH_INFO: `/v3/projects/1c7aaa02e0a2496cab5733c70c07c55d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:02.739208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7aaa02e0a2496cab5733c70c07c55d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:02.740401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 11:18:02.740401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Aug 09 11:18:02.740401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: Aug 09 11:18:02.750972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:02.771057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:02.864899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9e8f1-1727-444f-aecf-de069d2d33c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:02.919204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7aaa02e0a2496cab5733c70c07c55d', 'request_id': 'req-cf74320a-84d5-419a-9ef7-ddf6705b5bff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.935076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7aaa02e0a2496cab5733c70c07c55d', 'request_id': 'req-cf74320a-84d5-419a-9ef7-ddf6705b5bff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.962702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7aaa02e0a2496cab5733c70c07c55d', 'request_id': 'req-cf74320a-84d5-419a-9ef7-ddf6705b5bff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.976138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:02.976993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:02.977469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:02.977587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:02.987163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] The token cache is being invalidate because project 1c7aaa02e0a2496cab5733c70c07c55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:02.987771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.988948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf74320a-84d5-419a-9ef7-ddf6705b5bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:02.997566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1697] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:02 2022] DELETE /identity/v3/projects/1c7aaa02e0a2496cab5733c70c07c55d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:02.999872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10914-7064-450b-947c-a08250005751 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.001136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10914-7064-450b-947c-a08250005751 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.001711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10914-7064-450b-947c-a08250005751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.054809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:03.076377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-23c10914-7064-450b-947c-a08250005751 None None] MFA Rules not processed for user `f4da6dd860bb45d1a703a18c2512ffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:03.078517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23c10914-7064-450b-947c-a08250005751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.166291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:03.168179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fff34-f0ab-4fed-bc75-95937f78dac5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:03.189995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1698] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:03.199548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb480-3408-4867-9318-451233413e1f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.199548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb480-3408-4867-9318-451233413e1f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.199548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eb480-3408-4867-9318-451233413e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.244036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1699] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:03.245802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.254285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.254651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.262501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ae7eb480-3408-4867-9318-451233413e1f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:03.263268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1700] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 9 11:18:03 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:03.266541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae7eb480-3408-4867-9318-451233413e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.267056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:03.275007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.292443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1701] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:03.293940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709b90a-1d7c-46a9-a5a5-9ada381823d8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.295916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709b90a-1d7c-46a9-a5a5-9ada381823d8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.296200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709b90a-1d7c-46a9-a5a5-9ada381823d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.340094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2709b90a-1d7c-46a9-a5a5-9ada381823d8 None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:03.347730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2709b90a-1d7c-46a9-a5a5-9ada381823d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.365918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1702] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:03.367978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.368364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.368552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.429814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:03.431155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.453327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1703] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:03.456521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:03.463769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.500040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb19782-9af1-4a9b-af84-fac2c3bab470 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:03.512194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:03.513370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.515010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.515356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.545622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:03.549436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:03.577772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439247f-417d-42ac-b5f2-5c194cf80626 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:03.584064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:03.584975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.585473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.585944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.588165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.588377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1704] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:03.601267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:03.631239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:03.680915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ae6c8-18f7-4121-bb9a-267914cdb523 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:03.699234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:03.700788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.701053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.701325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.701973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:03.702790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:03.703952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c3cfd-0742-485c-8108-d2b6da1abe35 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:03.712209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:03.714720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83f06a-db6d-4ef9-b1c7-223be6991d66 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:03.752805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1705] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:03.754985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.755412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.755831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.759600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1706] 10.222.0.27 () {54 vars in 961 bytes} [Tue Aug 9 11:18:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:03.760587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:03.763099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.767122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1707] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:03.768513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.768797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.769066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.804881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48141e-bb1a-4e7e-b83b-7c7ac784d37c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:03.821798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:03.822538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:03.822922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:03.823574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] PATH_INFO: `/v3/users/2a9775cefaf24e6c805069fddf2ff369` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:03.829436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9775cefaf24e6c805069fddf2ff369)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:03.830978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:03.855880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:03.856885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:03.961819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9775cefaf24e6c805069fddf2ff369'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:03.975726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9775cefaf24e6c805069fddf2ff369'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:03.993708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9775cefaf24e6c805069fddf2ff369'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:03.994838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9775cefaf24e6c805069fddf2ff369'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.023820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9775cefaf24e6c805069fddf2ff369'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.039272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1708] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:03 2022] DELETE /identity/v3/users/2a9775cefaf24e6c805069fddf2ff369 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:04.045794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.049670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.052668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1709] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Aug 9 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:04.057575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.059315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.083383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b27e4-274d-48c9-9a16-6d9f374f842f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:04.171142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:04.213486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fb1e6-e5fe-42a9-93b2-8911efe9f16b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:04.228327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:04.229670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:04.230680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:04.231259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:04.232103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:04.235218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:04.235346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:04.236940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:04.237229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:04.237502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:04.250661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.284961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1710] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:04 2022] POST /identity/v3/projects => generated 405 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:04.286765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.290664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.343986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:04.475164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:04.482814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d0d49-2121-47d6-8bc9-44415ea59474 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:04.489900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e3fde-c3ca-4db1-b8e0-737fd8ab5cc8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:04.506730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:04.509018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:04.509970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:04.511228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] PATH_INFO: `/v3/projects/33404d64cb6a470fa3d1ad5ff116e7eb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:04.514138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=33404d64cb6a470fa3d1ad5ff116e7eb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:04.517427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1711] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:04.519424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:04.520675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:04.521060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:04.521981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:04.574774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:04.576119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.594792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1712] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:04.597909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.605520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33404d64cb6a470fa3d1ad5ff116e7eb', 'request_id': 'req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.613476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33404d64cb6a470fa3d1ad5ff116e7eb', 'request_id': 'req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.640301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33404d64cb6a470fa3d1ad5ff116e7eb', 'request_id': 'req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.660534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] The token cache is being invalidate because project 33404d64cb6a470fa3d1ad5ff116e7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:04.660982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.661891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:04.667999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1713] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:04 2022] DELETE /identity/v3/projects/33404d64cb6a470fa3d1ad5ff116e7eb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:04.667999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.668238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:04.771453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4054626-2110-4a0a-b602-84f2d20e1f18 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:04.823773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b3d0-e994-4cdd-a8a1-4fa1810305eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:04.838148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:04.840244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:04.840564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:04.843007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:04.856961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:04.863205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:04.937865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1714] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:04 2022] POST /identity/v3/users => generated 303 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:04.939241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:04.942570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.009167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.011956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.013311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.013899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.023328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.043056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.044862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.078662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1715] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:05.079765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:05.081440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.094232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:05.099457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c18f4-569f-432f-abb3-3c0eb0484987 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.107830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242009a-e0c6-4e73-acc9-f6d17673eb4d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.108927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.109762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.110494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.110981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] PATH_INFO: `/v3/users/2ac365959d7044faa3e0a09e4294ab20` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.112946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac365959d7044faa3e0a09e4294ab20)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.117647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.121145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.121544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.121825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.122093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.126565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.127633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:05.157444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.158967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.185548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1716] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:05.186979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:05.196906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.206228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac365959d7044faa3e0a09e4294ab20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:05.210110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac365959d7044faa3e0a09e4294ab20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:05.229686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac365959d7044faa3e0a09e4294ab20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:05.229934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f2e8-25e1-4175-998a-ffb7e044826b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.248592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.248592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.248899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.249132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.255487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac365959d7044faa3e0a09e4294ab20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:05.256064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.256681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac365959d7044faa3e0a09e4294ab20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:05.285243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.286232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1717] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:05 2022] DELETE /identity/v3/users/2ac365959d7044faa3e0a09e4294ab20 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:05.290921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:05.291283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.294494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:05.315721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1718] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:05.562448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715016-d912-4b0f-b443-3e13531f829f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.577191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.577877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.578148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.578409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.579081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.584903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.611589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1719] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:05.615863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:05.657641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72c736-8565-464e-90c6-88060b93cdd4 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.670550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.671372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.671746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.672112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.673019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.676495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1771440-c473-4e11-adc5-2150bdd5b233 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.720048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1720] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:05.722962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:05.740093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162ec54-a92b-4937-bfcb-ed0f23c76239 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:05.757075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:05.759442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.759926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.761699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] PATH_INFO: `/v3/projects/d22e7b8798c94ac19853ae669f03bd96/users/c8d694a87fe348b69160c7454c2b822c/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.823097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22e7b8798c94ac19853ae669f03bd96, user_id=c8d694a87fe348b69160c7454c2b822c, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:05.831540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:05.873175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1721] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:05 2022] PUT /identity/v3/projects/d22e7b8798c94ac19853ae669f03bd96/users/c8d694a87fe348b69160c7454c2b822c/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:05.887878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:05.889109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:05.889602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:05.995276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] MFA Rules not processed for user `c8d694a87fe348b69160c7454c2b822c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:05.997236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:06.171994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:06.175056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:06.228634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1722] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:06.233219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:06.466728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:06.472021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1c28a-fe6f-45e8-ad40-85e47bd938b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:06.515536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:06.516209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:06.516515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:06.517207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] PATH_INFO: `/v3/projects/7443ad1e03504e999842e1bf42fdf454` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:06.521022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7443ad1e03504e999842e1bf42fdf454)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:06.523110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:06.546037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67f83-4cf2-4fbd-af0d-ca7e986eacf9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:06.579056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:06.579705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:06.579981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:06.580253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:06.580935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:06.591079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:06.664203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1723] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:06 2022] POST /identity/v3/projects => generated 405 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:06.665988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:06.667997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:06.693669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7443ad1e03504e999842e1bf42fdf454', 'request_id': 'req-4564d6d8-7837-4970-af5e-f47e647c26cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:06.720956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7443ad1e03504e999842e1bf42fdf454', 'request_id': 'req-4564d6d8-7837-4970-af5e-f47e647c26cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:06.735349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:06.766297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859226-6b00-4b43-8e77-42d785ade063 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:06.774560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7443ad1e03504e999842e1bf42fdf454', 'request_id': 'req-4564d6d8-7837-4970-af5e-f47e647c26cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:06.817293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] The token cache is being invalidate because project 7443ad1e03504e999842e1bf42fdf454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:06.817600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:06.819445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:06.822081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1724] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:06 2022] DELETE /identity/v3/projects/7443ad1e03504e999842e1bf42fdf454 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:06.822881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:06.827384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:06.903544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:06.904213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:06.904521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:06.904847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] PATH_INFO: `/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:06.907205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b3bf2454e142fcab203d9ba33ed54c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:06.925200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:07.105117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564d6d8-7837-4970-af5e-f47e647c26cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:07.109120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b3bf2454e142fcab203d9ba33ed54c', 'request_id': 'req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.122006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b3bf2454e142fcab203d9ba33ed54c', 'request_id': 'req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.150634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b3bf2454e142fcab203d9ba33ed54c', 'request_id': 'req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.172967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] The token cache is being invalidate because project b3b3bf2454e142fcab203d9ba33ed54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:07.173241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.175410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.178106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1725] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:06 2022] DELETE /identity/v3/projects/b3b3bf2454e142fcab203d9ba33ed54c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:07.179072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:07.185311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.258592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:07.259211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.259559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.260002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.260916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:07.263317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f824c5-0572-4a73-a478-23191ca10446 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:07.304649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1726] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:06 2022] POST /identity/v3/users => generated 309 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:07.305899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.306790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.307079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.335656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:07.337091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.354525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1727] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:07.363110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:07.363110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.376831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569124-9194-4f2a-869f-054af9b2c3da None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:07.385382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:07.386018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.386469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.386744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.387328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd507d-3ab8-4679-a9ca-fc0caaffd8ee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:07.387517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:07.391905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f644-fe58-4f5b-a48a-557442b08866 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:07.405330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:07.406923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1728] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:07.408811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2203820-379b-4133-98af-f49111526ffe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.411223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.411349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2203820-379b-4133-98af-f49111526ffe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.411430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.411505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2203820-379b-4133-98af-f49111526ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.411578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] PATH_INFO: `/v3/users/2e32546bc48d4d9e9b936f995cae00ee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.416923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e32546bc48d4d9e9b936f995cae00ee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:07.420185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:07.430686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:07.439960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a2203820-379b-4133-98af-f49111526ffe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:07.441812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a2203820-379b-4133-98af-f49111526ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.453983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1729] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:07.457278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2203820-379b-4133-98af-f49111526ffe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:07.481706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2203820-379b-4133-98af-f49111526ffe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:07.499550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:07.508118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:07.511330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:07.511900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:07.513082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] PATH_INFO: `/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8/users/1dacdc8eb3684f33b932f54738a5c331/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:07.574956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e32546bc48d4d9e9b936f995cae00ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.584825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e32546bc48d4d9e9b936f995cae00ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.588491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=676f9fdd36c6435bb807cfaa8c8536a8, user_id=1dacdc8eb3684f33b932f54738a5c331, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:07.606710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e32546bc48d4d9e9b936f995cae00ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.607930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e32546bc48d4d9e9b936f995cae00ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.613012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:07.651084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e32546bc48d4d9e9b936f995cae00ee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:07.661910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1730] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:07 2022] PUT /identity/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8/users/1dacdc8eb3684f33b932f54738a5c331/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:07.666123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:07.670466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1731] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:07 2022] DELETE /identity/v3/users/2e32546bc48d4d9e9b936f995cae00ee => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:07.670466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:07.676810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.688672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:07.922244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfffb6-708a-456f-80c3-31faedb3a38d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.011036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85ece0-410d-4483-b761-2e6c723646c3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.021401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.023666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.023666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.023666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.024200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.025891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.063764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1732] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:08.065371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.080351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.081123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.081440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.081709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.084395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:08.084882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.090247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.106286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41854d-490c-4506-9d6d-967d23026177 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.125172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1733] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:08.125870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.126117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.126561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.126840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.127187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] PATH_INFO: `/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8/users/1dacdc8eb3684f33b932f54738a5c331/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.134995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=676f9fdd36c6435bb807cfaa8c8536a8, user_id=1dacdc8eb3684f33b932f54738a5c331, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.141306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262b871-b639-4232-a0fc-94eb473912bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.157469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfff8-2b8d-4b6d-9ead-070510c3387e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.167901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1734] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:08 2022] PUT /identity/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8/users/1dacdc8eb3684f33b932f54738a5c331/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:08.180354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.180648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.180996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.274084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] MFA Rules not processed for user `1dacdc8eb3684f33b932f54738a5c331`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:08.278136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:08.295473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.296226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.296447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.297103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.297682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.300724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.368754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1735] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:08 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:08.375166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.382043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:08.483509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f817798-5f94-42dc-8f37-a9cbe1560743 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.486401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1736] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:08.494547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.495349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.495631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.495944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.496696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.497892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.500563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:08.501678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.521142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:08.549913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1737] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:08 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:08.554731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.573914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59eee3-394d-4946-9f3f-dfd079a1f38c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.580990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.583043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.583477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.583825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.584688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.587595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.600324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1738] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:08.604471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.618794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaa74f-9100-4f99-9bfc-041d11388d64 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.626009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.626566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.626960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.628022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.628852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.631889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.649349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1739] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:08.653109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:08.675307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600fe5a-c7e9-44b4-9f8a-e51d7db7ad76 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.688211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.690490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.691069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.691619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] PATH_INFO: `/v3/projects/4125cc1a5ddc44b595a90d91e7beead2/users/801d2d157bae4f66808e65eed1f416b6/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.716312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93cfb9-0f65-4f81-aad8-66ae2f9654bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:08.772557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4125cc1a5ddc44b595a90d91e7beead2, user_id=801d2d157bae4f66808e65eed1f416b6, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:08.777952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3446bf-147b-4003-8f67-e2baa8998fb0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:08.824210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1740] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:08 2022] PUT /identity/v3/projects/4125cc1a5ddc44b595a90d91e7beead2/users/801d2d157bae4f66808e65eed1f416b6/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:08.826909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.827453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.827884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.861778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:08.862771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:08.863218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:08.863616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:08.887377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] MFA Rules not processed for user `801d2d157bae4f66808e65eed1f416b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:08.889461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:09.003668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.005876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.032238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1741] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:08 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:09.045234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1742] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:09.049719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.070087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2d39a-4aed-487b-9af1-a803a43c4898 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.214975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:09.215618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.215893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.216159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.216887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.224619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.259524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:09.275658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.306203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1743] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:09 2022] POST /identity/v3/projects => generated 373 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:09.307490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.311719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10428f9-f674-4830-b140-e551d86c9240 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.402630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:09.469326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:09.472378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.472912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.473024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] PATH_INFO: `/v3/users/c8d694a87fe348b69160c7454c2b822c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.481174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68ecbf-668c-4c7d-9cf2-d28bd5b0c587 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.482204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d694a87fe348b69160c7454c2b822c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.485556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.493323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:09.494942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.495904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.496512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.497957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.502531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.559283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1744] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:09 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:09.562946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.599962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343535c-4d30-4468-9379-276a3bc41bfb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.610553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d694a87fe348b69160c7454c2b822c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:09.611496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d694a87fe348b69160c7454c2b822c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:09.613831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:09.614477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.615095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.615095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.615626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.618072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.622129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d694a87fe348b69160c7454c2b822c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:09.655588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d694a87fe348b69160c7454c2b822c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:09.656969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8bfe0243-b899-4eed-b4be-9d165ff30c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d694a87fe348b69160c7454c2b822c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:09.657095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1745] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:09.657944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.666378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:09.690423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1746] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:09 2022] DELETE /identity/v3/users/c8d694a87fe348b69160c7454c2b822c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:09.690423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.690423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.690423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.695834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f392b9-343e-46e6-9e80-642493a52229 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.809979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:09.811374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:09.838281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:09.839075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.839348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.839618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] PATH_INFO: `/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6/users/5bc2d2c369ac4b15b00a3886649b3f17/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.881181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8d68fcbe5e45a9b0ff95fc558a99d6, user_id=5bc2d2c369ac4b15b00a3886649b3f17, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:09.889734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7792a-bf48-4e3e-8085-e093ef995025 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:09.947881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1747] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:09 2022] PUT /identity/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6/users/5bc2d2c369ac4b15b00a3886649b3f17/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:09.949412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:09.949760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:09.950063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:09.968428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1748] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:09.970499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:09.997542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:09.998508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15e7c9-4af0-45d3-a247-ac9c9a56fd8f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:09.998946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:10.104201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:10.125481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:10.141700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1749] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:10.208879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:10.211083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:10.216547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:10.218423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.218763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.219011 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.220122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:10.225909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:10.264167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1750] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:10.266642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:10.328218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a2a62-b2fb-4790-a4d6-7d385c968fc7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:10.344864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d931814-2207-4f8a-bcd8-08c4d1bd8146 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:10.345291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:10.346191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.346606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.347036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] PATH_INFO: `/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6/users/5bc2d2c369ac4b15b00a3886649b3f17/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.349306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8d68fcbe5e45a9b0ff95fc558a99d6, user_id=5bc2d2c369ac4b15b00a3886649b3f17, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:10.352531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:10.354573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:10.357549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.359618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.359913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.380086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1751] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:10 2022] PUT /identity/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6/users/5bc2d2c369ac4b15b00a3886649b3f17/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:10.381417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:10.390828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:10.403938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:10.407721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0649-268c-4802-95d5-79adeadea9f4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:10.445111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:10.447067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.447358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.447632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.494453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] MFA Rules not processed for user `5bc2d2c369ac4b15b00a3886649b3f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:10.495864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:10.633393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812249bb-164c-4595-913c-a9d290151eec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:10.663404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:10.664047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.664431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.664736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.727021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1753] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:10.728636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:10.732606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:10.817004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:10.818710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e75cda-0b04-4553-a67d-135e94041ea6 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:10.831381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1754] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:18:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:18:10.878847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34099a1c-9dca-48c2-b74a-95c3954fd717 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:10.926642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:10.927970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:10.931968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:10.933375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:10.984550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:10.986840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:11.033232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1755] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:11.112416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e738d-fec0-4cf2-9047-e9f60d075d96 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:11.112774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e738d-fec0-4cf2-9047-e9f60d075d96 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:11.113136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e738d-fec0-4cf2-9047-e9f60d075d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:11.151578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-556e738d-fec0-4cf2-9047-e9f60d075d96 None None] MFA Rules not processed for user `2e6543b377f14949a6919af4e4e187b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:11.156535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-556e738d-fec0-4cf2-9047-e9f60d075d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:11.211340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:11.282931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1756] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:11.600827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:11.603688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:11.707786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:11.826645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e718a0c-ee9c-4fcf-a0bc-0da6e470e694 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:11.842129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:11.843929 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:11.844211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:11.844515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] PATH_INFO: `/v3/projects/ce088013ad2d4146b5b43e66e0db538c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:11.848290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce088013ad2d4146b5b43e66e0db538c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:11.854674 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:12.049335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce088013ad2d4146b5b43e66e0db538c', 'request_id': 'req-23f25242-f481-4ae8-898e-b8db7b4d735b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.105668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.106235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce088013ad2d4146b5b43e66e0db538c', 'request_id': 'req-23f25242-f481-4ae8-898e-b8db7b4d735b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.106340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.106340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.130615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce088013ad2d4146b5b43e66e0db538c', 'request_id': 'req-23f25242-f481-4ae8-898e-b8db7b4d735b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.148470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] MFA Rules not processed for user `f4da6dd860bb45d1a703a18c2512ffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:12.152553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.187481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] The token cache is being invalidate because project ce088013ad2d4146b5b43e66e0db538c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:12.187481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.187481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23f25242-f481-4ae8-898e-b8db7b4d735b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.187481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1757] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:11 2022] DELETE /identity/v3/projects/ce088013ad2d4146b5b43e66e0db538c => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:12.195626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad6f0-d1ee-406e-9c78-128061b82391 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.196869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad6f0-d1ee-406e-9c78-128061b82391 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.197161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad6f0-d1ee-406e-9c78-128061b82391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.276965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92cad6f0-d1ee-406e-9c78-128061b82391 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:12.281111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92cad6f0-d1ee-406e-9c78-128061b82391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.396783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1758] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:12.396783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:12.398024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.542211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1759] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:12.543727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.544013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.544294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.606624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:12.607751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.639415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1760] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:12.640443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:12.676890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ad029-8d14-4291-9823-9ee6d68f9b9c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:12.698103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:12.703546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.703835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.704295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.704971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:12.705350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:12.707627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff466c-075b-4f67-bb6a-97e6ebc51c5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:12.723840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1761] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:12.723840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.723840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.724214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.735176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699cc1e-2eac-4e61-b198-fcc974d9d58f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:12.770989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:12.770989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:12.770989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:12.770989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] PATH_INFO: `/v3/users/801d2d157bae4f66808e65eed1f416b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:12.772531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:12.773792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.782790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] RBAC: Authorizing `identity:delete_user(user_id=801d2d157bae4f66808e65eed1f416b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:12.784943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:12.933346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801d2d157bae4f66808e65eed1f416b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.934924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:12.936854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801d2d157bae4f66808e65eed1f416b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.952196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801d2d157bae4f66808e65eed1f416b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.974245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1762] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Aug 9 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:12.975117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:12.984349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:12.998289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801d2d157bae4f66808e65eed1f416b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:12.999615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a024df6c-a520-4717-b7c8-a01f1718e8de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801d2d157bae4f66808e65eed1f416b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:13.019852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:13.031958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9e355-6e19-483a-86d8-4af6cec1cc16 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.038630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.038832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1763] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:12 2022] DELETE /identity/v3/users/801d2d157bae4f66808e65eed1f416b6 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:13.038916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.042049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.042049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.042049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.042787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.045382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.046045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.047047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.090669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:13.105192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1764] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:12 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:13.106397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:13.120962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.244917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa5040-7cb0-49ad-a7d6-3546883ef216 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.251325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.252131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.252599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.253032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.253863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.257855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.307134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1765] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:13 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:13.308248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.311419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:13.371756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1766] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:13.373329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.399460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eda9fc-9b32-4c93-af98-a785fff59ec6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.417259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.417893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.418164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.418426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.420111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.422418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.455141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1767] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:13.456373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.465331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:13.486156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5d410-5866-418e-9448-7ae48cf04268 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.492714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e3b17-a53d-4771-a54f-4cf9ee7cf931 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.497368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.500356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.501451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.502604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.503513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.504186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.504492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.505188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.522750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.524367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.533867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1768] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:13.534873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.537465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.543138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.548741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2158a0-0b01-4b58-9beb-7a7a4b6f741e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.557418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.558431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.558727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.559122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.560393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.565710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88217fc0-02b7-471a-bf49-48fef7dc91b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.573688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1769] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:13.576211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1770] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:13.580799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:13.598050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c0768-e203-47e2-97f9-178dc3678f03 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:13.612184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:13.613619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.614174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.614590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] PATH_INFO: `/v3/projects/57336cd78c5e421fb3863677b06f5cb8/users/6864d5073f564d8b99d27c0158567fc3/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.660866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57336cd78c5e421fb3863677b06f5cb8, user_id=6864d5073f564d8b99d27c0158567fc3, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:13.676817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:13.734970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1771] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:13 2022] PUT /identity/v3/projects/57336cd78c5e421fb3863677b06f5cb8/users/6864d5073f564d8b99d27c0158567fc3/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:13.755008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:13.755646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:13.755646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:13.904519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] MFA Rules not processed for user `6864d5073f564d8b99d27c0158567fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:13.906701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:14.086148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:14.094584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:14.126183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:14.255269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1772] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:14.260094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:14.380446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c9aae-56a9-447f-9697-4e17a6ab5aa1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:14.388095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:14.389498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:14.389985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:14.390366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] PATH_INFO: `/v3/projects/d22e7b8798c94ac19853ae669f03bd96` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:14.391972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22e7b8798c94ac19853ae669f03bd96)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:14.396676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:14.498303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3790-8740-40ea-8ff3-ec258bebc9c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:14.536067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22e7b8798c94ac19853ae669f03bd96', 'request_id': 'req-c2a4076d-3900-4bad-9b78-5e96e75c6e62'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.558934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:14.579910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22e7b8798c94ac19853ae669f03bd96', 'request_id': 'req-c2a4076d-3900-4bad-9b78-5e96e75c6e62'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.596405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22e7b8798c94ac19853ae669f03bd96', 'request_id': 'req-c2a4076d-3900-4bad-9b78-5e96e75c6e62'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.614768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] The token cache is being invalidate because project d22e7b8798c94ac19853ae669f03bd96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:14.615245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.616021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.630074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1773] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:14 2022] DELETE /identity/v3/projects/d22e7b8798c94ac19853ae669f03bd96 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:14.630658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:14.632123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:14.679571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:14.681265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:14.681798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:14.682336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:14.687377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:14.692737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:14.718706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4076d-3900-4bad-9b78-5e96e75c6e62 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:14.728541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:14.731509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:14.732254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:14.733217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] PATH_INFO: `/v3/users/1dacdc8eb3684f33b932f54738a5c331` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:14.736352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dacdc8eb3684f33b932f54738a5c331)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:14.743904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:14.758732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1774] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:14 2022] POST /identity/v3/projects => generated 385 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:14.763492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:14.768143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:14.870838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dacdc8eb3684f33b932f54738a5c331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.872724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dacdc8eb3684f33b932f54738a5c331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.890272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dacdc8eb3684f33b932f54738a5c331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.916518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dacdc8eb3684f33b932f54738a5c331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.917439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dacdc8eb3684f33b932f54738a5c331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:14.955362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1775] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:14 2022] DELETE /identity/v3/users/1dacdc8eb3684f33b932f54738a5c331 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:15.061768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9db6-41d0-403a-ae0a-c411322eddaf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:15.099725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:15.251509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:15.252176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.252444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.252708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.253357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:15.255660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:15.305395 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1776] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:14 2022] POST /identity/v3/users => generated 299 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:15.309593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:15.311448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:15.334438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd659a5-2e6d-43d3-9346-b7337f2cb69b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:15.347424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:15.348439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.348942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.349395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.350449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:15.354925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:15.384189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1777] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:15.388120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:15.401209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c4d4-86f3-4376-8427-0bbda594679a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:15.415810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:15.417168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.417684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.418199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] PATH_INFO: `/v3/projects/078898ffcf864516a7d75bc4799079ab/users/1c73c11583b14cf1ba30af53bfe6b9ac/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.472183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=078898ffcf864516a7d75bc4799079ab, user_id=1c73c11583b14cf1ba30af53bfe6b9ac, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:15.475519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:15.510098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1778] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:15 2022] PUT /identity/v3/projects/078898ffcf864516a7d75bc4799079ab/users/1c73c11583b14cf1ba30af53bfe6b9ac/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:15.514237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:15.536699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1816-2340-4a83-ac56-715a2ddcd9b8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:15.648056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:15.660964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:15.664761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.665059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.665344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.669851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:15.671477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f777b0-fcc7-4f77-8c99-ba747f3d42bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:15.713879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1779] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:15.714991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:15.747879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016a59-ed54-4c90-96b8-b9c6b018b64b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:15.767467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:15.768123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.768430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.770042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] PATH_INFO: `/v3/projects/078898ffcf864516a7d75bc4799079ab/users/1c73c11583b14cf1ba30af53bfe6b9ac/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.801978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=078898ffcf864516a7d75bc4799079ab, user_id=1c73c11583b14cf1ba30af53bfe6b9ac, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:15.801978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:15.853525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:15.853789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:15.854059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:15.857108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1780] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:15 2022] PUT /identity/v3/projects/078898ffcf864516a7d75bc4799079ab/users/1c73c11583b14cf1ba30af53bfe6b9ac/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:15.961159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] MFA Rules not processed for user `1c73c11583b14cf1ba30af53bfe6b9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:15.962993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:16.051750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:16.055532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:16.270899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1781] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:16.279631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:16.281705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:16.397380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e854fe5-f19a-475f-adcd-e07d922cd4aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:16.416516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:16.417338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:16.417456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:16.417774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] PATH_INFO: `/v3/projects/4125cc1a5ddc44b595a90d91e7beead2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:16.422828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4125cc1a5ddc44b595a90d91e7beead2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:16.424887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:16.431150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8231e-719c-4cf4-b5f0-237084a9b492 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:16.443666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:16.446058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:16.446540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:16.447625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:16.508999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:16.510522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:16.549239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1782] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:16.605888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4125cc1a5ddc44b595a90d91e7beead2', 'request_id': 'req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:16.607964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:16.627437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4125cc1a5ddc44b595a90d91e7beead2', 'request_id': 'req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:16.638053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4219f-ef45-4e32-b34d-6aae320f50e1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:16.663942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4125cc1a5ddc44b595a90d91e7beead2', 'request_id': 'req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:16.711804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:16.724501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] The token cache is being invalidate because project 4125cc1a5ddc44b595a90d91e7beead2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:16.726901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:16.729907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:16.731999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1783] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:16 2022] DELETE /identity/v3/projects/4125cc1a5ddc44b595a90d91e7beead2 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:16.739068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:16.741146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:16.887868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:16.888771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:16.888946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:16.889262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] PATH_INFO: `/v3/users/6864d5073f564d8b99d27c0158567fc3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:16.891258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6864d5073f564d8b99d27c0158567fc3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:16.899549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:16.946737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:16.995637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bc36d-dd76-4ea0-a144-eb9a9729ab73 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:17.019622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:17.020508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.023585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.026737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6864d5073f564d8b99d27c0158567fc3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.027202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] PATH_INFO: `/v3/users/5bc2d2c369ac4b15b00a3886649b3f17` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.027734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6864d5073f564d8b99d27c0158567fc3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.031028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc2d2c369ac4b15b00a3886649b3f17)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:17.039170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:17.041514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6864d5073f564d8b99d27c0158567fc3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.070800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6864d5073f564d8b99d27c0158567fc3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.074452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9162b2fc-8694-4b68-9ce2-f26b6cfec552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6864d5073f564d8b99d27c0158567fc3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.108940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1784] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:16 2022] DELETE /identity/v3/users/6864d5073f564d8b99d27c0158567fc3 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:17.111622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daf711-6248-42fd-9b42-29a8adb2e8fd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.111988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daf711-6248-42fd-9b42-29a8adb2e8fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.112543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43daf711-6248-42fd-9b42-29a8adb2e8fd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.114669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1785] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:18:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:17.117112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.117470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.118608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.159007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:17.160478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:17.164090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc2d2c369ac4b15b00a3886649b3f17'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.203740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc2d2c369ac4b15b00a3886649b3f17'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.227734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc2d2c369ac4b15b00a3886649b3f17'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.228426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc2d2c369ac4b15b00a3886649b3f17'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.291634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47e25b72-df32-4a07-b35d-b723db2c3e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc2d2c369ac4b15b00a3886649b3f17'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:17.306239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1786] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:16 2022] DELETE /identity/v3/users/5bc2d2c369ac4b15b00a3886649b3f17 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:17.307936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.311184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.311669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.331646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1787] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:17.356746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:17.358296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:17.407298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:17.415714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:17.483246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387cc0f-aea2-4fdb-ba51-1d1dff6e8f85 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:17.630947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:17.631582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.631856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.632361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.653942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:17.767882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1788] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:17.771203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:17.780357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:17.822220 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:17.823798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:17.841493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ba355-b58e-41b7-bf01-c84b20881805 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:17.864982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:17.867795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:17.873035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:17.874686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] PATH_INFO: `/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:17.878754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=676f9fdd36c6435bb807cfaa8c8536a8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:17.880511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1789] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:17.892306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:17.943768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:18.093627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:18.096175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:18.114486 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeca46-a0a8-4006-91b8-eb0083abb2cb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:18.133124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676f9fdd36c6435bb807cfaa8c8536a8', 'request_id': 'req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:18.170096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676f9fdd36c6435bb807cfaa8c8536a8', 'request_id': 'req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:18.208183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676f9fdd36c6435bb807cfaa8c8536a8', 'request_id': 'req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:18.254104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] The token cache is being invalidate because project 676f9fdd36c6435bb807cfaa8c8536a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:18.256881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:18.259810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2337b578-bba1-44cc-b4d1-db5dcdcfb5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:18.266449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1790] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:17 2022] DELETE /identity/v3/projects/676f9fdd36c6435bb807cfaa8c8536a8 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:18.272864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44af5-1103-4b35-8776-32377b302bdc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.273381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44af5-1103-4b35-8776-32377b302bdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.274172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44af5-1103-4b35-8776-32377b302bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.332192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-77d44af5-1103-4b35-8776-32377b302bdc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:18.337402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77d44af5-1103-4b35-8776-32377b302bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:18.537461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:18.538203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.538662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.538951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.542176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1791] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:18.549007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3422d-37e4-44ad-b4c3-ad7eb91122d6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.549292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3422d-37e4-44ad-b4c3-ad7eb91122d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.549520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3422d-37e4-44ad-b4c3-ad7eb91122d6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.556054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1792] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:18:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:18.558833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.559819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:18.561081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.562828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.617262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:18.619203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cb9a-0e21-43eb-8fce-00c8ab501d6f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:18.641411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:18.643848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:18.667612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1793] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:18.671483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1794] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:18.671718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.672016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.672313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.673173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:18.689789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db87526-73bb-4d07-8bb2-f8553edfedf1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:18.698252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:18.698252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.698252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.698252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.698639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:18.698784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:18.700883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:18.725872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1795] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:18.728108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:18.754384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c368-d107-4155-9ac2-1339dfbbec59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:18.760989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:18.767072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:18.768058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:18.768768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:18.769434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:18.770626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:18.773461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:18.774101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:18.790080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1796] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:18.853830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:18.854321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1797] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:18 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:18.868974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f378ef-598f-47bb-b8ee-de2ff3a5d292 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:19.076254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:19.077970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.078253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.079782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.081185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:19.087836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:19.138006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1798] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:18 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:19.143807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:19.163451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b849b-50ce-4955-9b8f-135be08ac4cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:19.177551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:19.178211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.178556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.178903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.180037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:19.183816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:19.219181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1799] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:19.223804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:19.238758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0033df0-c8b9-43bd-89bc-a6c52e66d378 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:19.252034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:19.254471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.256886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.257233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.262932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:19.266756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466287d8-32ce-40cb-9061-ef2ac0c20eb5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:19.272074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:19.291260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1800] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:19.296836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:19.317955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38b686-f05c-4bdc-bdf2-1e1da180025c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:19.331314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:19.331553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.333774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.334198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] PATH_INFO: `/v3/projects/1034ad4f5c4a4a1799287dd5fac1990b/users/da205616148249eeb02438a11ea14de6/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.376814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1034ad4f5c4a4a1799287dd5fac1990b, user_id=da205616148249eeb02438a11ea14de6, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:19.385622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:19.425357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1801] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:19 2022] PUT /identity/v3/projects/1034ad4f5c4a4a1799287dd5fac1990b/users/da205616148249eeb02438a11ea14de6/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:19.442302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.442688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.443097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.510060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] MFA Rules not processed for user `da205616148249eeb02438a11ea14de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:19.512066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:19.616393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:19.618586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:19.665619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:19.776627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1802] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:19.781456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:19.824966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52267d8b-80da-4db0-9611-596b9790b880 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:19.844193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:19.845070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:19.845487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:19.846578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] PATH_INFO: `/v3/projects/57336cd78c5e421fb3863677b06f5cb8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:19.847973 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=57336cd78c5e421fb3863677b06f5cb8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:19.850216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:19.985066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f28fc-adaf-4d34-a398-9402e3be3e75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.002456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57336cd78c5e421fb3863677b06f5cb8', 'request_id': 'req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.012655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57336cd78c5e421fb3863677b06f5cb8', 'request_id': 'req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.024142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57336cd78c5e421fb3863677b06f5cb8', 'request_id': 'req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.041819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] The token cache is being invalidate because project 57336cd78c5e421fb3863677b06f5cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:20.042276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.043182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.047926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1803] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:19 2022] DELETE /identity/v3/projects/57336cd78c5e421fb3863677b06f5cb8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:20.048830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.050427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.125942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.126615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.126892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.127159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.127902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.130023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.208248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1804] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:19 2022] POST /identity/v3/projects => generated 387 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:20.208248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.208248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.225769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d43c6-e14c-42e4-9f5b-f7e4893317cd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.259755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:20.322223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:20.387328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c25cab-6caa-4311-943b-8e7dea72c22e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.394947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.395998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.396502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.397045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.404504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.404786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.405860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.405860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.407390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] PATH_INFO: `/v3/users/1c73c11583b14cf1ba30af53bfe6b9ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.424239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c73c11583b14cf1ba30af53bfe6b9ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.428861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.455141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.457006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.486012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1805] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:20.487231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.489994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.533150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2baabf-4b48-4bb7-81b9-c71b9f2458fc service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.554130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.555741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.557041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.557397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.558186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.572769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c73c11583b14cf1ba30af53bfe6b9ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.574413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c73c11583b14cf1ba30af53bfe6b9ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.577698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.593544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c73c11583b14cf1ba30af53bfe6b9ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.618352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c73c11583b14cf1ba30af53bfe6b9ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.620563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c73c11583b14cf1ba30af53bfe6b9ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:20.644551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1806] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:20 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:20.647812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.652118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.658217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1807] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:20 2022] DELETE /identity/v3/users/1c73c11583b14cf1ba30af53bfe6b9ac => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:20.661374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.667448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:20.707544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f79ffd-a65b-48d9-9431-1d7074268e82 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.832379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57cf8-1002-4f3c-b9ba-95fe45930deb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:20.860398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.862498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.863063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.863571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] PATH_INFO: `/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.877113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:20.880021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:20.880841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:20.880956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:20.881646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.885903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8d68fcbe5e45a9b0ff95fc558a99d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:20.888412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.902370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:20.970857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1808] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:20.971878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:20.978341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.002969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0c6f-4b5f-4476-a67a-d23c28cedc35 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:21.007916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8d68fcbe5e45a9b0ff95fc558a99d6', 'request_id': 'req-bdeb3c84-352b-4b4d-8719-48166ed5bef4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.019038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8d68fcbe5e45a9b0ff95fc558a99d6', 'request_id': 'req-bdeb3c84-352b-4b4d-8719-48166ed5bef4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.038125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8d68fcbe5e45a9b0ff95fc558a99d6', 'request_id': 'req-bdeb3c84-352b-4b4d-8719-48166ed5bef4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.071342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:21.082504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] The token cache is being invalidate because project ed8d68fcbe5e45a9b0ff95fc558a99d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:21.083488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.085272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.090659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1809] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:20 2022] DELETE /identity/v3/projects/ed8d68fcbe5e45a9b0ff95fc558a99d6 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:21.092358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:21.100357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.140932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:21.141594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.141900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.142221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] PATH_INFO: `/v3/users/df814b36a7c642aea0fd072cba28732a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.148738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df814b36a7c642aea0fd072cba28732a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:21.150728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:21.237007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:21.297893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3c84-352b-4b4d-8719-48166ed5bef4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:21.309504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df814b36a7c642aea0fd072cba28732a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.310799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df814b36a7c642aea0fd072cba28732a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.314398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:21.315154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.315426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.317160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] PATH_INFO: `/v3/projects/ae786f65bddd4ddea95088a37b08fae6/users/9cd1078bca4f42f4b812e4b5951f3169/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.338874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df814b36a7c642aea0fd072cba28732a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.374361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae786f65bddd4ddea95088a37b08fae6, user_id=9cd1078bca4f42f4b812e4b5951f3169, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:21.380672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698913ba-edcd-4c6c-9989-e9c0b35731b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:21.381218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df814b36a7c642aea0fd072cba28732a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.384523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df814b36a7c642aea0fd072cba28732a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:21.430320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1810] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:21 2022] PUT /identity/v3/projects/ae786f65bddd4ddea95088a37b08fae6/users/9cd1078bca4f42f4b812e4b5951f3169/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:21.433418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e5b4e-fe16-4b54-9006-efce9e151c18 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.434668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e5b4e-fe16-4b54-9006-efce9e151c18 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.435009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e5b4e-fe16-4b54-9006-efce9e151c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.435894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1811] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:20 2022] DELETE /identity/v3/users/df814b36a7c642aea0fd072cba28732a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:21.436840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:21.443358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.491278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf6e5b4e-fe16-4b54-9006-efce9e151c18 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:21.495004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf6e5b4e-fe16-4b54-9006-efce9e151c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.751421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1f4a5-5899-45a3-a324-d090a0fef95b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:21.765659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1812] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:21.765659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.765659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.765659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.766366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:21.771326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.771855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.772434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.773536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:21.778534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9288e28-f72d-4b82-8e07-c5d17706034d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:21.828276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1813] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:21.829658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.829954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.831158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.832287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:21.833515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.857308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1814] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:21.858157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:21.895695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:21.902164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.906370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e5129-65e7-4670-adc9-df61b6feb6f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:21.926859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:21.927551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:21.927819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:21.939149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] PATH_INFO: `/v3/projects/ae786f65bddd4ddea95088a37b08fae6/users/9cd1078bca4f42f4b812e4b5951f3169/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:21.939248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1815] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:21.939248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:21.942351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:21.944378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae786f65bddd4ddea95088a37b08fae6, user_id=9cd1078bca4f42f4b812e4b5951f3169, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:21.960208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.020585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1816] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:21 2022] PUT /identity/v3/projects/ae786f65bddd4ddea95088a37b08fae6/users/9cd1078bca4f42f4b812e4b5951f3169/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:22.022477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.084146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:22.112859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcff3c0-07c3-40eb-ac42-9593e41c65a7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.248574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.253799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.259135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.259457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.287812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.289474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c67dce-f86f-4bb3-971f-3a77e37ebe12 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.296850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36c9a-48a8-437a-9d4d-b6d66d16d7b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.312706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1817] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:22.313893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.314696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.314696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.314884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.315615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.315735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.315836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.316038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:22.316384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.317988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.368318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1818] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:22.372007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.393518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] MFA Rules not processed for user `9cd1078bca4f42f4b812e4b5951f3169`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:22.399860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:22.403003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263188-67be-4016-b55b-1958936276ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.442890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.443519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.443853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.444247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.451633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.454023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.545217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1819] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:22 2022] POST /identity/v3/projects => generated 363 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:22.546534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.588065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1820] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:22.596947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.600354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:22.601139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:22.723089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067749d2-05d5-445b-8e25-18be66ad6339 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.741049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.741920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.742488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.742841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.743531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.745993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.746814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f4506-64f2-4030-9add-89fda4fc9183 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.806225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1821] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:22 2022] POST /identity/v3/users => generated 282 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:22.808400 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.825390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:22.836560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.836831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618035b0-a798-4c6c-8aec-8cb210f69bdf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.836933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.837105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.837334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.853647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.855491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.855946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.856424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.857344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.859042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.892182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1822] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:22.893647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:22.921324 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d37d7a-cf48-41df-872c-5f9cb2cca804 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:22.938117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:22.938655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:22.938922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:22.939270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:22.939931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.942357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.964721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:22.967447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07271e-8754-4717-a8de-64f45437576d service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:22.979838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1823] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:22.982574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.016035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1824] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:23.017918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e5554-8835-43b7-9986-ac0005fad2ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.044140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.045168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.045582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.045923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] PATH_INFO: `/v3/projects/70a38e28e27e41ad8c2a782802db0b27/users/ee3f841759464313a11d8f832074ca8e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.108783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a38e28e27e41ad8c2a782802db0b27, user_id=ee3f841759464313a11d8f832074ca8e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.113924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.162106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1825] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:22 2022] PUT /identity/v3/projects/70a38e28e27e41ad8c2a782802db0b27/users/ee3f841759464313a11d8f832074ca8e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:23.181169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.181169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.182282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.234140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.267102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] MFA Rules not processed for user `ee3f841759464313a11d8f832074ca8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:23.269796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:23.412537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305bd2f8-59e6-493e-8a71-c20314ca815c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.429414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.432441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.433249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.433972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] PATH_INFO: `/v3/users/da205616148249eeb02438a11ea14de6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.436701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=da205616148249eeb02438a11ea14de6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.442660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.464947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1826] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:23.469594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.496562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01f7ed-e9f9-4142-9d82-5c4f00bc2b92 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.510048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.510722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.511075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.511450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.513662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.517561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.566520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da205616148249eeb02438a11ea14de6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:23.580235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da205616148249eeb02438a11ea14de6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:23.586183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1827] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:23 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:23.587404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.589294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:23.597596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da205616148249eeb02438a11ea14de6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:23.598784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da205616148249eeb02438a11ea14de6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:23.601441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:23.621276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da205616148249eeb02438a11ea14de6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:23.634103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1828] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:23 2022] DELETE /identity/v3/users/da205616148249eeb02438a11ea14de6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:23.635193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.637894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:23.831484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cd900-5d55-4874-9721-a0eb11ceae9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.838585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:23.862630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b81c44-5156-492b-bb59-5eca30d2bdfe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.871904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.872974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.873399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.873976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.874821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.876983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1888c-128a-497e-9819-3fd212aca792 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.908333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1829] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:23 2022] POST /identity/v3/users => generated 288 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:23.909965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.910430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.911006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.928951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.929829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.930201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.930737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] PATH_INFO: `/v3/projects/078898ffcf864516a7d75bc4799079ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.943429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:23.943833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=078898ffcf864516a7d75bc4799079ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.945886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:23.947730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.956348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1830] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:23.957635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:23.970028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175d5f-a3f2-481a-8646-939d432dfc79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:23.977288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:23.978239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:23.978741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:23.979007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:23.979796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:23.981499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:23.999127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1831] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:23.999127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.003022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.013113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078898ffcf864516a7d75bc4799079ab', 'request_id': 'req-e2aeebc2-8094-4c0e-8778-d5f05572e362'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.021649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078898ffcf864516a7d75bc4799079ab', 'request_id': 'req-e2aeebc2-8094-4c0e-8778-d5f05572e362'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.029413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078898ffcf864516a7d75bc4799079ab', 'request_id': 'req-e2aeebc2-8094-4c0e-8778-d5f05572e362'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.049198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] The token cache is being invalidate because project 078898ffcf864516a7d75bc4799079ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:24.049767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.051057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.053236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1832] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:23 2022] DELETE /identity/v3/projects/078898ffcf864516a7d75bc4799079ab => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:24.055173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.057033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.083721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf977-1fd1-4267-aa27-63d932acc4d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.152316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aeebc2-8094-4c0e-8778-d5f05572e362 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.157496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.158420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.158793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.159155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] PATH_INFO: `/v3/projects/36d867b9404744f8a5a3d8451da0c01a/users/91e692d73ecc468e8add547bc0d0c651/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.167008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.168046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.168578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.168986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.172769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.176876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d867b9404744f8a5a3d8451da0c01a, user_id=91e692d73ecc468e8add547bc0d0c651, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.180179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3b4a0-fa53-4d2e-8433-8aea4ca74424 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.192593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.195789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.204569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1833] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:24 2022] PUT /identity/v3/projects/36d867b9404744f8a5a3d8451da0c01a/users/91e692d73ecc468e8add547bc0d0c651/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:24.204569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f437a-83c3-493b-8b11-4f4a7679992d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.205333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f437a-83c3-493b-8b11-4f4a7679992d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.205709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f437a-83c3-493b-8b11-4f4a7679992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.232586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1834] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:24.234750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.251097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6c4f437a-83c3-493b-8b11-4f4a7679992d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:24.252249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c4f437a-83c3-493b-8b11-4f4a7679992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.257411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903204-f3e5-44c8-a914-07e7ac3cabb5 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.279828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:24.285639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:24.349733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1835] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:24.358534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.359145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.359421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.359686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.360144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.360366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.360623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.361054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.361970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.373581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1836] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:24.379064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.388374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:24.389836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.392293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf57c5-29d6-4573-906f-2bf56968d38b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.412290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.412923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1837] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:24.413447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.414012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.414326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] PATH_INFO: `/v3/projects/36d867b9404744f8a5a3d8451da0c01a/users/91e692d73ecc468e8add547bc0d0c651/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.419103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.421226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d867b9404744f8a5a3d8451da0c01a, user_id=91e692d73ecc468e8add547bc0d0c651, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.424959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.431099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729fcc3-3ba3-4b24-a968-cf3456c226e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.437876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.438927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.439349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.440076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.441037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:24.441474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.444024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d91d7-a933-4bbf-8d74-d0d3a57797b0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.448802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1838] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:24 2022] PUT /identity/v3/projects/36d867b9404744f8a5a3d8451da0c01a/users/91e692d73ecc468e8add547bc0d0c651/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:24.452363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.457940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.464849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1839] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:18:24.466151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.466510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.466856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.512722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] MFA Rules not processed for user `91e692d73ecc468e8add547bc0d0c651`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:24.514351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.578863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7fd6-eaf2-454a-adcc-104669c071de admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.585328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.586258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.586647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.587014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] PATH_INFO: `/v3/projects/3e2a289f595c433a9448f0f73947a39b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.589923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2a289f595c433a9448f0f73947a39b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.593925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.611063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1840] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:24.612108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.634441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c1f12-9d42-424d-ad82-cd67522d3aaa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.640675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.641416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.641791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.642201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.642934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.645563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.682535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1841] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:24 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:24.683414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.687547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.712522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2a289f595c433a9448f0f73947a39b', 'request_id': 'req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.720475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2a289f595c433a9448f0f73947a39b', 'request_id': 'req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.737877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2a289f595c433a9448f0f73947a39b', 'request_id': 'req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.766957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] The token cache is being invalidate because project 3e2a289f595c433a9448f0f73947a39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:24.766957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.767946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:24.768059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7725b4-efd9-4387-a510-fdeef0d9595e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.775398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1842] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:24 2022] DELETE /identity/v3/projects/3e2a289f595c433a9448f0f73947a39b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:24.776511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.781410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.794012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:24.843923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:24.865294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.865926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.866203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.866538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.869561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.885958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1dcf9-c15d-4ca7-ae17-d37642d571a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.891478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.892094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.892404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.892681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.893342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.895233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.930496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1843] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:24 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:24.930496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.934118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:24.957030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176600c-295f-4bdb-9ac8-32fbc03120af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:24.963455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:24.964333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:24.965094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:24.965094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] PATH_INFO: `/v3/users/3b872ef01ae345f8b541c8b0fcd724c3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:24.967140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b872ef01ae345f8b541c8b0fcd724c3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.969518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.982318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:24.983868 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:24.994862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1844] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:24.995684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:24.997077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.012688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502d9a5-63dc-4d74-bb68-57797d51ab16 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.020413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.021083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.021330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.021595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] PATH_INFO: `/v3/projects/1034ad4f5c4a4a1799287dd5fac1990b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.022989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1034ad4f5c4a4a1799287dd5fac1990b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.025106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.037408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b872ef01ae345f8b541c8b0fcd724c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.049263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b872ef01ae345f8b541c8b0fcd724c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.058774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b872ef01ae345f8b541c8b0fcd724c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.059645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b872ef01ae345f8b541c8b0fcd724c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.078233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b872ef01ae345f8b541c8b0fcd724c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.078581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1034ad4f5c4a4a1799287dd5fac1990b', 'request_id': 'req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.084612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1034ad4f5c4a4a1799287dd5fac1990b', 'request_id': 'req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.085540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1845] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:24 2022] DELETE /identity/v3/users/3b872ef01ae345f8b541c8b0fcd724c3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:25.086627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.087729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.095375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1034ad4f5c4a4a1799287dd5fac1990b', 'request_id': 'req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.123481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] The token cache is being invalidate because project 1034ad4f5c4a4a1799287dd5fac1990b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:25.123892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.124692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.126651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1846] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:24 2022] DELETE /identity/v3/projects/1034ad4f5c4a4a1799287dd5fac1990b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:25.130295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.132898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.204786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7161f-5978-4a2b-b143-d724b32485f8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.218246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.219412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.220291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.220889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.221873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.223647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:25.225362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.231580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82d9b6-ef5e-44b7-8d8a-2cb6ba8e4c4a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.240037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.240989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.241490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.241995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] PATH_INFO: `/v3/users/9cd1078bca4f42f4b812e4b5951f3169` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.243740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd1078bca4f42f4b812e4b5951f3169)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.245833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.269982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1847] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:25.270712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.274287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.291202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3310d6-b67a-4ce0-a45e-b284d6d8783e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.296625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.297712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.298250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.298774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.299670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.301747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.324115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1848] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:25.326205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.339354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d0d57-fdcb-4d56-a553-419ebafed38a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.345201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.348959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.349364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.349751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] PATH_INFO: `/v3/projects/e6a58ec6aa4641dfb34b20fd5059947d/users/d4356749706b4d2c8940f320ec1c8b5e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.359394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd1078bca4f42f4b812e4b5951f3169'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.360066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd1078bca4f42f4b812e4b5951f3169'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.365197 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:25.372030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a58ec6aa4641dfb34b20fd5059947d, user_id=d4356749706b4d2c8940f320ec1c8b5e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.377053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787ae55-76f2-4b3b-b968-1ef0c8d3e02d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.385144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd1078bca4f42f4b812e4b5951f3169'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.400363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1849] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:25 2022] PUT /identity/v3/projects/e6a58ec6aa4641dfb34b20fd5059947d/users/d4356749706b4d2c8940f320ec1c8b5e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:25.411451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.411451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.411642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.415701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd1078bca4f42f4b812e4b5951f3169'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.418738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd1078bca4f42f4b812e4b5951f3169'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.439060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1850] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:25 2022] DELETE /identity/v3/users/9cd1078bca4f42f4b812e4b5951f3169 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:25.482028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] MFA Rules not processed for user `d4356749706b4d2c8940f320ec1c8b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:25.484022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.581150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.583290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.601348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1851] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:25.602268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.603640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.719325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc51a1-5cd5-441d-b836-cfac798248e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.725054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.726033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.726442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.731366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc11954-20eb-4f97-8c88-be98842bbcb3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.731770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] PATH_INFO: `/v3/users/ee3f841759464313a11d8f832074ca8e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.737522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.739326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.739326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.739509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3f841759464313a11d8f832074ca8e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.739594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.740053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.742663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.742756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.776119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1852] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:25 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:25.777964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.780259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.805015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3f841759464313a11d8f832074ca8e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.806418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3f841759464313a11d8f832074ca8e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.815950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3f841759464313a11d8f832074ca8e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.829161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3f841759464313a11d8f832074ca8e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.830392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3f841759464313a11d8f832074ca8e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:25.837808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:25.850769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1853] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:25 2022] DELETE /identity/v3/users/ee3f841759464313a11d8f832074ca8e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:25.853009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:25.857240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:25.863288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b238-1f87-410e-a280-f7e13092590c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.872476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:25.941135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1723fd-bdab-4020-a405-25b2eff98d08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:25.946758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.947515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.947886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.948398 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.949207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.952719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9b24c-4e5c-4b5a-a845-f7560f472655 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.953632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:25.954295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.954573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.954847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] PATH_INFO: `/v3/projects/69e556169d9a49fca0cbcd7a082d2def` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:25.956007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e556169d9a49fca0cbcd7a082d2def)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:25.959237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:25.983777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1854] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:25 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:25.983777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:25.983777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:25.983777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.007526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.009465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.027923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e556169d9a49fca0cbcd7a082d2def', 'request_id': 'req-48bbd893-fb66-4279-9890-8d9a6080c7bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.038082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e556169d9a49fca0cbcd7a082d2def', 'request_id': 'req-48bbd893-fb66-4279-9890-8d9a6080c7bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.050691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e556169d9a49fca0cbcd7a082d2def', 'request_id': 'req-48bbd893-fb66-4279-9890-8d9a6080c7bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.073215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] The token cache is being invalidate because project 69e556169d9a49fca0cbcd7a082d2def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:26.073958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.074872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.078162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1855] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:25 2022] DELETE /identity/v3/projects/69e556169d9a49fca0cbcd7a082d2def => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:26.078991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.080933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.092130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1856] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:26.093337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.099237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbd893-fb66-4279-9890-8d9a6080c7bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.106213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.106840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.107324 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.107439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.108364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.109872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.112580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090640-90c1-4a29-9d10-7b8cc23d6717 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.123534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c63d47-1110-4fca-96ec-c37357c83d7a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.129103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.129765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.130038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.130310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] PATH_INFO: `/v3/projects/ae786f65bddd4ddea95088a37b08fae6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.131775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae786f65bddd4ddea95088a37b08fae6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.133812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.138700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1857] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:26.140748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.141195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.141626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.167142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.168429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.178239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1858] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:26.179541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.180871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.198420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae786f65bddd4ddea95088a37b08fae6', 'request_id': 'req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.204814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae786f65bddd4ddea95088a37b08fae6', 'request_id': 'req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.211902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae786f65bddd4ddea95088a37b08fae6', 'request_id': 'req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.230221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] The token cache is being invalidate because project ae786f65bddd4ddea95088a37b08fae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:26.230438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.231234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:26.233743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1859] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:26 2022] DELETE /identity/v3/projects/ae786f65bddd4ddea95088a37b08fae6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:26.234372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.236547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.250180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:26.253743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d8f0-7320-4c1e-b46c-ae12ee863d9f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.311465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:26.326763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.327594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485a8d-2d41-4c31-a80c-fe4a6f05ec20 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.327737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.328170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.328581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.331520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.333658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.334436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.334712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.335026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] PATH_INFO: `/v3/projects/6aefbd7d5dc64dddbe03646126d0e314/users/3edfd7e2d11e4c278d6c961f22fa0feb/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.346640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.349406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edcc1c-471a-42ff-a81b-4ecd75886d91 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.364004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aefbd7d5dc64dddbe03646126d0e314, user_id=3edfd7e2d11e4c278d6c961f22fa0feb, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.365500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1860] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:26.367795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.368139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57f4a7-d819-40e0-af44-537016434178 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.368677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.368936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.394088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1861] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:26 2022] PUT /identity/v3/projects/6aefbd7d5dc64dddbe03646126d0e314/users/3edfd7e2d11e4c278d6c961f22fa0feb/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:26.395316 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d42e1-ef28-4c79-9e2c-cbae4f8cd26b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.395595 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d42e1-ef28-4c79-9e2c-cbae4f8cd26b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.395860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d42e1-ef28-4c79-9e2c-cbae4f8cd26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.407028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.409309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.424511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c23d42e1-ef28-4c79-9e2c-cbae4f8cd26b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.425721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c23d42e1-ef28-4c79-9e2c-cbae4f8cd26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.479810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1862] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:26.481248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.491101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a78a7-ba45-432c-b3b1-6386962eb7bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.496528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.497249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.497610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.497941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.499198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.500947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.512193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1863] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:26.513078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.518196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1864] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:26.520203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.520675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.520926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.524425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359a3ad-f51c-4313-8d2d-a000e01fe4bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.529968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.530585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.530860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.531133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.531818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:26.532117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.533773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.545684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.547183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.548028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1865] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:26.549146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.564321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352b79-f66c-4283-ac26-34ceec07e0c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.564842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1866] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:26.566816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.575038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.575974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.576448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.576928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] PATH_INFO: `/v3/projects/6aefbd7d5dc64dddbe03646126d0e314/users/3edfd7e2d11e4c278d6c961f22fa0feb/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.583270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e638-27ca-4ed7-b682-e17a58666e1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.585017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aefbd7d5dc64dddbe03646126d0e314, user_id=3edfd7e2d11e4c278d6c961f22fa0feb, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.588922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.589862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.590528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.590930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.591300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.592785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.600681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5b142-9be9-40fa-ae7a-1af7a57922fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.614765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1867] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:26 2022] PUT /identity/v3/projects/6aefbd7d5dc64dddbe03646126d0e314/users/3edfd7e2d11e4c278d6c961f22fa0feb/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:26.615714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.640673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1868] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:26.642097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a93a-919c-4656-899d-f4c796617788 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.642383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a93a-919c-4656-899d-f4c796617788 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.642655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a93a-919c-4656-899d-f4c796617788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.705007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a359a93a-919c-4656-899d-f4c796617788 None None] MFA Rules not processed for user `3edfd7e2d11e4c278d6c961f22fa0feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:26.706556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a359a93a-919c-4656-899d-f4c796617788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.750454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9132f-6afa-431c-969b-e977b05883ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.776991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:26.808374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1869] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:26.812188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359a93a-919c-4656-899d-f4c796617788 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.821463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:26.834266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.835008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.835339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.835934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.836561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:26.836994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.838751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.854370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1870] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:26.854370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.856103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:26.926197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359a93a-919c-4656-899d-f4c796617788 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.935608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.936227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.936718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.936860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.937414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:26.939436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:26.969134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb7ffe-433b-4cd4-922b-6d35f998cc07 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:26.974791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:26.975442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:26.975753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:26.976025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:26.978618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1871] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:26.979620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:26.982707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.004131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9cc40-7815-45e9-ba46-a115c16e31fb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.014403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.015106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.015446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.015843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.016590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.019748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.074659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.077335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.111701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1872] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:26 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:27.113080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.117088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1873] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:27.118657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.148073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02257398-2ca5-453c-ab11-b06eb0442808 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.253301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.253756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.254480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.254864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.256608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.258871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.316360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1874] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:27 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:27.317856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.326913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.336318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074b027-1e56-4f0a-b8a7-9de1953a0290 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.342543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.343076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.343512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.343692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.344716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.346261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.356057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0cd99-f55e-4d55-8815-7ee67c4e2709 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.364399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1875] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:27.369015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.371638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.371822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.371921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.372189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] PATH_INFO: `/v3/projects/70a38e28e27e41ad8c2a782802db0b27` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.373795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a38e28e27e41ad8c2a782802db0b27)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.376595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.396886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9239620-4a8c-4e18-a6be-210878e01ace admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.421287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.422698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.424004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.424315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.425090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.427518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:27.428660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.448580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:27.450312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.457729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a38e28e27e41ad8c2a782802db0b27', 'request_id': 'req-93e36ffc-e795-4a94-befa-22fe6c3c81d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.466975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fba410-c175-479b-bfdb-82f1d768e549 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.475878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a38e28e27e41ad8c2a782802db0b27', 'request_id': 'req-93e36ffc-e795-4a94-befa-22fe6c3c81d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.493596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a38e28e27e41ad8c2a782802db0b27', 'request_id': 'req-93e36ffc-e795-4a94-befa-22fe6c3c81d3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.518125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] The token cache is being invalidate because project 70a38e28e27e41ad8c2a782802db0b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:27.518541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.519573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.524666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1877] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:27 2022] DELETE /identity/v3/projects/70a38e28e27e41ad8c2a782802db0b27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:27.526813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.530085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.537556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:27.581891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.583236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.583970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.584621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.585756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.590570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.639478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1878] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:27.641213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.652206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.677227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c2138-846b-4a5e-b918-4e600457dcb9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.687546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.688501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.688501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.688760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] PATH_INFO: `/v3/users/91e692d73ecc468e8add547bc0d0c651` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.694536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e692d73ecc468e8add547bc0d0c651)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.696555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.743830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36ffc-e795-4a94-befa-22fe6c3c81d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.753115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.754410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.755097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.755594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.759296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.763766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.789594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e692d73ecc468e8add547bc0d0c651'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.790175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1879] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:27.791931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e692d73ecc468e8add547bc0d0c651'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.792189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.795756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.823575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e692d73ecc468e8add547bc0d0c651'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.834338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73cd6f-11cd-4acd-9604-4ad934564f2c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:27.843187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e692d73ecc468e8add547bc0d0c651'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.843921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e692d73ecc468e8add547bc0d0c651'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:27.845772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:27.848341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.849443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.853987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] PATH_INFO: `/v3/projects/e278ae8b0fa34455905b9368c5f220e8/users/ca1143bb11574b94b534e06bdcddf484/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:27.874848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1880] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:27 2022] DELETE /identity/v3/users/91e692d73ecc468e8add547bc0d0c651 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:27.874848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:27.877543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:27.905846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e278ae8b0fa34455905b9368c5f220e8, user_id=ca1143bb11574b94b534e06bdcddf484, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:27.913497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c909b48-8886-40c8-8039-d4437cf03cc2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:27.950675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:27 2022] PUT /identity/v3/projects/e278ae8b0fa34455905b9368c5f220e8/users/ca1143bb11574b94b534e06bdcddf484/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:27.961150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:27.962550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:27.962848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.016302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] MFA Rules not processed for user `ca1143bb11574b94b534e06bdcddf484`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:28.017938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c406d-49fc-4f74-b1fb-73ed0e24b8a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.018171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.122388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:28.137619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1882] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:28.138377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.153248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.154241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.154682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.155633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] PATH_INFO: `/v3/projects/d83b297ee9174f1bb4350eb6b43bba45/users/2e1cbae660c64cfba3e581ef9784c485/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.181072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83b297ee9174f1bb4350eb6b43bba45, user_id=2e1cbae660c64cfba3e581ef9784c485, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.186359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac078d-c4ed-4078-aeca-19e7146e968a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.189136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b712718-9e02-447d-90ed-702344e3f34c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.215371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.216022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.218968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.219489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.221785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.223265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1883] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:27 2022] PUT /identity/v3/projects/d83b297ee9174f1bb4350eb6b43bba45/users/2e1cbae660c64cfba3e581ef9784c485/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:28.225295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.226883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.226883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.227156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.268031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] MFA Rules not processed for user `2e1cbae660c64cfba3e581ef9784c485`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:28.269296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.296555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:28.326369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1884] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:28.330102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.403847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1885] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:28.404920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.406491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.442868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2c19a-8cca-440d-839b-1e5a1cd06d66 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.452110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.452806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.453398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.453398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.456524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.463127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.513260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1886] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:28.514251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.527268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f042ed6-a0fc-4a12-a14c-fb44af45b2df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.534156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.534900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.535944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.536229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.536903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.540780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.541524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9d522-5427-4035-8680-0411126b20e9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.547896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.548659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.548998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.549338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] PATH_INFO: `/v3/projects/36d867b9404744f8a5a3d8451da0c01a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.550728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d867b9404744f8a5a3d8451da0c01a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.553406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.580769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1887] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:28.581764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.626737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d867b9404744f8a5a3d8451da0c01a', 'request_id': 'req-b21962d3-8e78-4141-b4b9-55769ad67ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:28.638229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d867b9404744f8a5a3d8451da0c01a', 'request_id': 'req-b21962d3-8e78-4141-b4b9-55769ad67ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:28.648175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d867b9404744f8a5a3d8451da0c01a', 'request_id': 'req-b21962d3-8e78-4141-b4b9-55769ad67ab0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:28.663244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:28.673320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] The token cache is being invalidate because project 36d867b9404744f8a5a3d8451da0c01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:28.673825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:28.676048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:28.676180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d7724-5c64-400c-8a6b-35c7dc0e251a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.677985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1888] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:28 2022] DELETE /identity/v3/projects/36d867b9404744f8a5a3d8451da0c01a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:28.681929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.688636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.780528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.781220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.781564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.781900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.782653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.784725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d34c9-aaa0-4ca9-bebf-921306ea805a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.788691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21962d3-8e78-4141-b4b9-55769ad67ab0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.793923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.794617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.795016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.795291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.795920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.798172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.798552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1889] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:28.800033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.800489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.800905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.811466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:28.826767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1890] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:28.828380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:28.830697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.830833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.832892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.847882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7038994-c754-4735-bb43-943f258bafac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.853138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1891] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:28.854272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.861389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.862088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.862363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.866008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754e837-476a-49ed-b729-c75fea2f7e73 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.868127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] PATH_INFO: `/v3/projects/d5ac3af3cae44166a153b49ed4aff68c/users/944d8ed8caa04367b1530f0a783c036b/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.871871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.872615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.872950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.873359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.874005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.875796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2891-49d5-485d-a958-a6cc47162bac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.892076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ac3af3cae44166a153b49ed4aff68c, user_id=944d8ed8caa04367b1530f0a783c036b, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:28.892076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:28.892784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1892] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:28.894194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.894904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.894904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:28.912112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1893] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:28 2022] PUT /identity/v3/projects/d5ac3af3cae44166a153b49ed4aff68c/users/944d8ed8caa04367b1530f0a783c036b/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:28.913390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:28.939180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:28.940533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:28.987219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269aab-fc25-405d-92fe-215b75dbf246 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:28.993507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:28.994577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:28.994920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:28.995312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] PATH_INFO: `/v3/projects/c0d51871d26c4c859a5d0ade43432446/users/b49a0e745411414fa519aff76f007ba8/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.023181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d51871d26c4c859a5d0ade43432446, user_id=b49a0e745411414fa519aff76f007ba8, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.031295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.075619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1895] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:29.075795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1895] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:28 2022] PUT /identity/v3/projects/c0d51871d26c4c859a5d0ade43432446/users/b49a0e745411414fa519aff76f007ba8/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:29.076437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.080236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.087332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cd265-6a70-4b12-bfc0-d61d0d6c1360 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.172792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:29.192718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.193358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.193799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.194088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.195020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.208512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.217445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c0ec7-9f64-4e93-8c0e-6daea51c9bb2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.231579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1896] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:29.231579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.236392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.237250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.237621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.238191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.239005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.241680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.260505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48161d2a-0c5f-4139-8e08-bf1f68329598 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.267006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1897] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:29.267301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.268132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.268345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.268469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.268804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.269538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:29.269910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.271592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.281464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3633543-5d27-4763-9a38-f28261f65829 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.286591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.287548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.288003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.288318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] PATH_INFO: `/v3/projects/d5ac3af3cae44166a153b49ed4aff68c/users/944d8ed8caa04367b1530f0a783c036b/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.291198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ac3af3cae44166a153b49ed4aff68c, user_id=944d8ed8caa04367b1530f0a783c036b, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.295335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.296467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1898] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:29.298176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.314876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1899] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:29 2022] PUT /identity/v3/projects/d5ac3af3cae44166a153b49ed4aff68c/users/944d8ed8caa04367b1530f0a783c036b/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:29.315811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.318318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ed82-4322-4073-97e9-3bf21fb3d54a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.406507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.407389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.407670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.407941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] PATH_INFO: `/v3/projects/c0d51871d26c4c859a5d0ade43432446/users/b49a0e745411414fa519aff76f007ba8/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.412111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d51871d26c4c859a5d0ade43432446, user_id=b49a0e745411414fa519aff76f007ba8, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.415173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a641c4-f8e6-48b3-a888-0550daa535d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.417987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e51f43-7650-405d-83c4-ceebc5c39578 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.424729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.425350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.425621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.426039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.426673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.428770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe108f34-e62f-4470-a941-eecdfa3c0a8d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.431925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1900] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:29 2022] PUT /identity/v3/projects/c0d51871d26c4c859a5d0ade43432446/users/b49a0e745411414fa519aff76f007ba8/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:29.433637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.434061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.434780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.445059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:29.463532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1901] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:29 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:29.465197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.465197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.465475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.468985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] MFA Rules not processed for user `944d8ed8caa04367b1530f0a783c036b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:29.470428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.502855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] MFA Rules not processed for user `b49a0e745411414fa519aff76f007ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:29.504168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.555277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1902] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:29.557555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.584792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1903] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:29.585871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.587774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.629716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdc752-17e7-4155-9690-14a6d30056f2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.635157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.635774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.636052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.636714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.636973 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.638937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.642169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:29.659699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a198cde-7408-46e1-ae00-a4881052df0f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.664355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.665296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.665639 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.665849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.666006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1904] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:29 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:29.667063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.667624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.668870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.681092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e69ecf-e14d-4abc-8ebd-053693e8be86 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.684739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.685451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.686114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.686114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.695266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.696975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.707029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1905] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:29.707673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.719556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0e67-a4ce-4303-8f9e-59ff22f84d6c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.733843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.734776 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.735282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.735717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.736672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.739387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27548c-7eff-48dd-b6a0-45b0852f53cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.739903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.741734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a81d1f-bf75-4828-b678-1024c5e5011a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.767833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1906] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:29.769599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5fdb9-53a4-470f-8122-0b1efa340346 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.770192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5fdb9-53a4-470f-8122-0b1efa340346 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.770728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5fdb9-53a4-470f-8122-0b1efa340346 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.772223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1907] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:18:29.778612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1908] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:29.782654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5fdb9-53a4-470f-8122-0b1efa340346 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.801733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.802015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.802280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.830115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5fdb9-53a4-470f-8122-0b1efa340346 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.848813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.852049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.852445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.853331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.864903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:29.869279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:29.870317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:29.871804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:29.899638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1909] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:29.900790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:29.921217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50248ba3-44c1-4755-bd1c-30e6df9c4b36 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:29.928151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:29.929132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:29.929464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:29.929830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:29.933259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.009074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:30.052186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1910] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:30.053796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.071871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.073516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543572e-5c12-4f51-9012-79d37ec363d5 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.083446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0806257-43d2-443d-bab6-9e0d9e00a0f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.085347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1911] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:30.107508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.108774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.109995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.110550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] PATH_INFO: `/v3/projects/60edf10f6c8b4be5bdd183799ea38e03/users/16a2e28855474d78be82c24bfcc6a3c2/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.150623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60edf10f6c8b4be5bdd183799ea38e03, user_id=16a2e28855474d78be82c24bfcc6a3c2, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.155301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.190619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1912] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:30 2022] PUT /identity/v3/projects/60edf10f6c8b4be5bdd183799ea38e03/users/16a2e28855474d78be82c24bfcc6a3c2/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:30.199502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.202006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.202369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.276591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] MFA Rules not processed for user `16a2e28855474d78be82c24bfcc6a3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:30.276941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.280301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.283762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.404616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1913] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:30.408697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.415124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8b7c-84f4-4c33-989b-a13f4f2227b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.430098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.440455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349104ab-f42d-4274-9589-da3352e8f1ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.440969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.440969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.440969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] PATH_INFO: `/v3/users/d4356749706b4d2c8940f320ec1c8b5e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.440969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4356749706b4d2c8940f320ec1c8b5e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.440969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.447850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.448827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.449546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.449996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.451038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.472328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.490800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:30.510856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4356749706b4d2c8940f320ec1c8b5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:30.528131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1914] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:30 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:30.528131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.528131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.529097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4356749706b4d2c8940f320ec1c8b5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:30.546788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4356749706b4d2c8940f320ec1c8b5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:30.547436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4356749706b4d2c8940f320ec1c8b5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:30.568212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4356749706b4d2c8940f320ec1c8b5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:30.585485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1915] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:30 2022] DELETE /identity/v3/users/d4356749706b4d2c8940f320ec1c8b5e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:30.586462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.588432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.648330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236821-65c3-4349-9424-b9beddb696bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.668617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:30.699540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c501e0-5d8b-49ab-a497-71a089a99b6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.709394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.710520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.711278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.711877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.713526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.716767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f480940-e0cc-4526-ab6a-2a0b988e339b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.747106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1916] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:30 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:30.748296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.748644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.749245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.759743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.761780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.761780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.761780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.764036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.777555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.779157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.781609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:30.782917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.791647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1917] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:30.792438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1918] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:30.792558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.803724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58d09-a6cc-4ddb-9ed2-12292f70dc49 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.804086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.805971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:30.809287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.809908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.810187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.810523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.811168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.812934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.826902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1919] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:30.831366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.843216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee5bcf-0108-49b8-963e-88db23c24516 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.847776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.848562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.848940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.849280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] PATH_INFO: `/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2/users/96a2d3c823a549abbda82b420716546f/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.866525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bcf375c96e47cab1bc7cd8bf4a3da2, user_id=96a2d3c823a549abbda82b420716546f, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:30.870330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:30.893446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad454cf9-68b8-4905-a403-e1d9f53418d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:30.896491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1920] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:30 2022] PUT /identity/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2/users/96a2d3c823a549abbda82b420716546f/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:30.901507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:30.974859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:30.975833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:30.976465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:30.977141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:30.990854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:31.025542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff915c6-e00d-4899-99f8-676c15772e7a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:31.030925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:31.031627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.031959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.032314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.033032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.034796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.050731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1921] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:31.051844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:31.064610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b75701-f86d-4118-9d0d-2c9d6961a465 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:31.069626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:31.070406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.071151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.071497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] PATH_INFO: `/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2/users/96a2d3c823a549abbda82b420716546f/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.074202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bcf375c96e47cab1bc7cd8bf4a3da2, user_id=96a2d3c823a549abbda82b420716546f, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.076820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.092714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.094184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f28c-7eb8-470f-8c7f-45024883e126 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.105618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1922] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:31.120800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1923] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:31 2022] PUT /identity/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2/users/96a2d3c823a549abbda82b420716546f/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:31.134730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.135083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.135563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.187949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] MFA Rules not processed for user `96a2d3c823a549abbda82b420716546f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:31.190343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:31.229907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:31.303227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1924] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:31.311231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:31.313311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:31.426977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02267ac-abbd-411c-83c1-e017efdc49af admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:31.434091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:31.435447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.436307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.436307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.451153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.453092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.484858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1925] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:31.550807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:31.552630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:31.627765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:31.639984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f164d38-96e5-446f-b732-dfe2361c2467 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:31.650200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:31.650963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.651303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.651638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.658073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:31.787456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4b4a-9ca7-4663-bb1b-c2192fc9484c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:31.797668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.799311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318214af-898a-4967-a875-114e7990a001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:31.799869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.800578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318214af-898a-4967-a875-114e7990a001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:31.801047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318214af-898a-4967-a875-114e7990a001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:31.801322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318214af-898a-4967-a875-114e7990a001 admin admin] PATH_INFO: `/v3/projects/e6a58ec6aa4641dfb34b20fd5059947d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:31.803090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318214af-898a-4967-a875-114e7990a001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a58ec6aa4641dfb34b20fd5059947d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:31.806732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318214af-898a-4967-a875-114e7990a001 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:31.813619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1926] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:31.909084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a58ec6aa4641dfb34b20fd5059947d', 'request_id': 'req-318214af-898a-4967-a875-114e7990a001'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:31.922278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a58ec6aa4641dfb34b20fd5059947d', 'request_id': 'req-318214af-898a-4967-a875-114e7990a001'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:31.937781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a58ec6aa4641dfb34b20fd5059947d', 'request_id': 'req-318214af-898a-4967-a875-114e7990a001'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:31.989519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] The token cache is being invalidate because project e6a58ec6aa4641dfb34b20fd5059947d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:31.990645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:31.992977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:31.995927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1927] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:31 2022] DELETE /identity/v3/projects/e6a58ec6aa4641dfb34b20fd5059947d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:32.000591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:32.019489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda57aa-2c15-4b9d-9165-9200ab44eee6 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:32.117247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:32.118538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:32.118619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:32.118950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] PATH_INFO: `/v3/users/3edfd7e2d11e4c278d6c961f22fa0feb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:32.120034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edfd7e2d11e4c278d6c961f22fa0feb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:32.121997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:32.126195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:32.219880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edfd7e2d11e4c278d6c961f22fa0feb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.220768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edfd7e2d11e4c278d6c961f22fa0feb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.232583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edfd7e2d11e4c278d6c961f22fa0feb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.252540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edfd7e2d11e4c278d6c961f22fa0feb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.254186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7175ed8-f54a-4d90-9b17-4f78be1dde73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edfd7e2d11e4c278d6c961f22fa0feb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.283785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1928] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:31 2022] DELETE /identity/v3/users/3edfd7e2d11e4c278d6c961f22fa0feb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:32.728319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:32.730094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:32.743747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318214af-898a-4967-a875-114e7990a001 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:32.788876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:32.805439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:32.806074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:32.806344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:32.806611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] PATH_INFO: `/v3/projects/6aefbd7d5dc64dddbe03646126d0e314` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:32.807792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aefbd7d5dc64dddbe03646126d0e314)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:32.809739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:32.859320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aefbd7d5dc64dddbe03646126d0e314', 'request_id': 'req-92de4de1-ac16-46c1-9325-cec5bd9b7020'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.865845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aefbd7d5dc64dddbe03646126d0e314', 'request_id': 'req-92de4de1-ac16-46c1-9325-cec5bd9b7020'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.876747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aefbd7d5dc64dddbe03646126d0e314', 'request_id': 'req-92de4de1-ac16-46c1-9325-cec5bd9b7020'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.889181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] The token cache is being invalidate because project 6aefbd7d5dc64dddbe03646126d0e314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:32.889497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.890202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-92de4de1-ac16-46c1-9325-cec5bd9b7020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:32.891739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1929] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:32 2022] DELETE /identity/v3/projects/6aefbd7d5dc64dddbe03646126d0e314 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:32.896897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c0902-4767-486e-9641-958861b30abe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:32.897342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c0902-4767-486e-9641-958861b30abe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:32.897518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c0902-4767-486e-9641-958861b30abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:32.917806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bd2c0902-4767-486e-9641-958861b30abe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:32.919062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bd2c0902-4767-486e-9641-958861b30abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:32.980857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1930] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:32.988625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:32.988907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:32.989155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.010649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:33.012501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:33.021538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1931] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:33.025963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c0902-4767-486e-9641-958861b30abe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.035737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c0902-4767-486e-9641-958861b30abe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.041038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.041659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.041924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.042203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.042863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:33.043161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.044780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.059808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1932] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:33.061687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.071216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dacfb9-77ae-4a2c-a78b-334b0dedc562 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.076621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.077210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.077481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.077742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.078391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.080344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.109902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1933] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:33 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:33.114270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.124981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318ad1d-0211-4ac1-a1ed-ca9bfd1a3b67 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.184740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.185541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.185916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.186187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.186820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.188766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.214345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:33.235096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1934] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:33 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:33.239414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.250799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52071dd4-ca73-49f4-a4e8-40057ff9719d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.255659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.256772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.257591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.257865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.258904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.260551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.274511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1935] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:33.275704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.285875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13687ac5-15b9-4253-85b2-df447ce7e3ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.290835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.291450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.291728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.292000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.292709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.294294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b9265-bedc-40f5-a50c-9fc5423d9ddc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.308335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1936] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:33.309609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.320747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4f7f-c4a5-48b6-b888-1660af44cc59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.325789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.326527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.327517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.327867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] PATH_INFO: `/v3/projects/fcb6c8f78f6446f1a50c383d57df5b82/users/90261602123b4a83a0e417bb2af51d3c/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.345056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb6c8f78f6446f1a50c383d57df5b82, user_id=90261602123b4a83a0e417bb2af51d3c, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.347947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.373047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1937] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:33 2022] PUT /identity/v3/projects/fcb6c8f78f6446f1a50c383d57df5b82/users/90261602123b4a83a0e417bb2af51d3c/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:33.384665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.384665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.384665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.420015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] MFA Rules not processed for user `90261602123b4a83a0e417bb2af51d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:33.422204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:33.512568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1938] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:33.517208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.612627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2249c-e574-41aa-9e3d-af850c35e699 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.618464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.619393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.619809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.620175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.620943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.623350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.657759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:33.676706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1939] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:33 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:33.681004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.717471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45496-2256-4259-b79b-746f7a233bff None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.809685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.810862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.810862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.811208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.811953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.814588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.846322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1940] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:33 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:33.850454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.866729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c3379-3787-46f0-8305-d4b05f84126e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.884664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.887097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.887394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.887939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.888644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.890479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.926509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1941] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:33.930618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.950046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852dfb8-110e-4394-9166-57d0eb1dac01 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.957312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.958122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.958572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.959015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:33.959902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:33.970042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0820e02-0ce3-429f-b3a4-a95656174204 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:33.975293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1942] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:33.979242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:33.991305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9775d44-64a9-4357-a2d2-6d1eeb8c6fbb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:33.998551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:33.999228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:33.999504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:33.999773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] PATH_INFO: `/v3/projects/ccdf68165a454860a88fad3f6489ec69/users/96e6d11694df431482ebd95dd7c11e40/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.038385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccdf68165a454860a88fad3f6489ec69, user_id=96e6d11694df431482ebd95dd7c11e40, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.041401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.062092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1943] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:33 2022] PUT /identity/v3/projects/ccdf68165a454860a88fad3f6489ec69/users/96e6d11694df431482ebd95dd7c11e40/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:34.074646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.075249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.075593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.111597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] MFA Rules not processed for user `96e6d11694df431482ebd95dd7c11e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:34.113851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:34.162022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:34.186255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1944] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:34.190878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.282759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb1e8-db53-4f9f-ab23-af17db303872 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.288928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.289863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.290446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.290907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.291852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.294342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.339334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1945] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:34 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:34.343251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.354553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edc7af-9a27-4c06-87ae-735c2b1bfe32 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.427327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.428185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.428656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.428924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.429689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.432306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.481782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1946] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:34 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:34.485798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.497848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82606a2-faaa-4f45-8da6-5be82fe2e3d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.502824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.503461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.503741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.504009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.504921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.506396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.523763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1947] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:34.525232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.535638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc960-fc13-47b6-8c9b-20910b2c8539 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.540480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.541203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.541541 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.541876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] PATH_INFO: `/v3/projects/ec620c18816f45fa84f24ffcc73745bc/users/cf71bbeaf0154b46a73bf7c46d3896ef/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.556756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec620c18816f45fa84f24ffcc73745bc, user_id=cf71bbeaf0154b46a73bf7c46d3896ef, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.559423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.574509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1948] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:34 2022] PUT /identity/v3/projects/ec620c18816f45fa84f24ffcc73745bc/users/cf71bbeaf0154b46a73bf7c46d3896ef/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:34.578586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.589056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfe06a-44fc-4908-ba15-47e414469e85 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.649559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.650363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.650638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.650905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.651555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.653288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c47031-06ce-4dd5-9678-b8437f0356d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.668300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1949] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:34.671579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.686581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da42ee9-8363-4b0f-9066-2d883b6d2ed5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.691908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.692780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.693064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.693339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] PATH_INFO: `/v3/projects/ec620c18816f45fa84f24ffcc73745bc/users/cf71bbeaf0154b46a73bf7c46d3896ef/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.695301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec620c18816f45fa84f24ffcc73745bc, user_id=cf71bbeaf0154b46a73bf7c46d3896ef, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.698225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:34.710562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1950] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:34 2022] PUT /identity/v3/projects/ec620c18816f45fa84f24ffcc73745bc/users/cf71bbeaf0154b46a73bf7c46d3896ef/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:34.721461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.721813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.722061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.753712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] MFA Rules not processed for user `cf71bbeaf0154b46a73bf7c46d3896ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:34.754923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:34.778986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:34.782493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.784744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:34.840998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1951] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:34.855015 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:34.856637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:34.857632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f78a85-4087-4e18-a9e5-dda82bf755f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.863790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.864784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.865118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.865359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.869440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:34.872252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f565f-ffb5-4267-b4de-4d875787d1ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:34.878331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:34.878331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:34.878331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:34.892222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:34.969779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:34.993189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:34.994952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:35.009758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1952] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:35.021215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:35.023186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e502ca-d7fe-4735-973b-be2a39eda0de service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:35.036500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1953] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:35.754000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:35.755408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:35.831101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a5d-fa79-42a2-8968-05494cdeef4f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:35.836687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:35.837401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:35.837717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:35.838000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] PATH_INFO: `/v3/users/2e1cbae660c64cfba3e581ef9784c485` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:35.839838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1cbae660c64cfba3e581ef9784c485)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:35.841460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:35.893621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1cbae660c64cfba3e581ef9784c485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:35.894294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1cbae660c64cfba3e581ef9784c485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:35.902507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1cbae660c64cfba3e581ef9784c485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:35.928466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1cbae660c64cfba3e581ef9784c485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:35.928466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1cbae660c64cfba3e581ef9784c485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:35.945696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1954] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:35 2022] DELETE /identity/v3/users/2e1cbae660c64cfba3e581ef9784c485 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:35.948831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:35.949899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:35.950378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:35.950743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:35.950837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:35.974199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:35.975544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.033355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3b991-84f5-460f-965f-faa9af0aad85 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.040781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.041681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.041998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.042789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] PATH_INFO: `/v3/users/16a2e28855474d78be82c24bfcc6a3c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.044427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a2e28855474d78be82c24bfcc6a3c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.046553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.047109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1955] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:36.056172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:36.058747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.060376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.113853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a2e28855474d78be82c24bfcc6a3c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.114501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a2e28855474d78be82c24bfcc6a3c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.123937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a2e28855474d78be82c24bfcc6a3c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.131014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80127a51-82bf-41d1-b9ab-8808e40198b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.135851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.136568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.136819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.137081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.137409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a2e28855474d78be82c24bfcc6a3c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.138152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-06224aad-831f-4246-a323-ddb814a98d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a2e28855474d78be82c24bfcc6a3c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.138877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.151520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1956] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:35 2022] DELETE /identity/v3/users/16a2e28855474d78be82c24bfcc6a3c2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:36.151793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.153171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.156510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.156771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.157031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.158880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:36.163867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1957] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:36.177520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:36.178724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.243703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1958] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:36.253653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.255082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.267223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f85f6-ff01-4970-817f-97f9dad5bfbf service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.330647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.331345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.331888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.332215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.345844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.350784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.360133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1959] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:36.452852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.454666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.471157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfeaac4-0f16-484e-8369-371f87806f64 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.501277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.502331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.502700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.503777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] PATH_INFO: `/v3/projects/d83b297ee9174f1bb4350eb6b43bba45` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.506288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83b297ee9174f1bb4350eb6b43bba45)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.508472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.594396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83b297ee9174f1bb4350eb6b43bba45', 'request_id': 'req-285583fe-b2a0-485e-b122-af27303d4208'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.600613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83b297ee9174f1bb4350eb6b43bba45', 'request_id': 'req-285583fe-b2a0-485e-b122-af27303d4208'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.609304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83b297ee9174f1bb4350eb6b43bba45', 'request_id': 'req-285583fe-b2a0-485e-b122-af27303d4208'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.629623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] The token cache is being invalidate because project d83b297ee9174f1bb4350eb6b43bba45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:36.630483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.631238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.634771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1960] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:36 2022] DELETE /identity/v3/projects/d83b297ee9174f1bb4350eb6b43bba45 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:36.638894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.650594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b93d55-8ce0-4100-a494-bd1de678d2e9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.693066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.694457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.705880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:36.732756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.734093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.734665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.734943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] PATH_INFO: `/v3/users/b49a0e745411414fa519aff76f007ba8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.738365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49a0e745411414fa519aff76f007ba8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.740656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.763293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583fe-b2a0-485e-b122-af27303d4208 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:36.769149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:36.770162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:36.770162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:36.770580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:36.772596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.817181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49a0e745411414fa519aff76f007ba8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.826501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49a0e745411414fa519aff76f007ba8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.840299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49a0e745411414fa519aff76f007ba8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.840982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49a0e745411414fa519aff76f007ba8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.862336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49a0e745411414fa519aff76f007ba8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:36.864163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:36.865928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:36.869875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1961] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:36 2022] DELETE /identity/v3/users/b49a0e745411414fa519aff76f007ba8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:36.876685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1962] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:36.897194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:36.898659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:36.950292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:37.009304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3fefe-9880-453e-bb0c-af1a11bee4f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.016991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.017774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.018054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.018466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] PATH_INFO: `/v3/projects/60edf10f6c8b4be5bdd183799ea38e03` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.020037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60edf10f6c8b4be5bdd183799ea38e03)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.022214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.097526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60edf10f6c8b4be5bdd183799ea38e03', 'request_id': 'req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.109187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60edf10f6c8b4be5bdd183799ea38e03', 'request_id': 'req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.126158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60edf10f6c8b4be5bdd183799ea38e03', 'request_id': 'req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.147071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] The token cache is being invalidate because project 60edf10f6c8b4be5bdd183799ea38e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:37.147365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.148072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.150181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1963] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:36 2022] DELETE /identity/v3/projects/60edf10f6c8b4be5bdd183799ea38e03 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:37.155117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:37.157118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.176078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b1d9a-f5c6-4914-bb63-5d8e36c3aa50 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.258163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.259229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.259649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.259849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] PATH_INFO: `/v3/users/96a2d3c823a549abbda82b420716546f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.261532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a2d3c823a549abbda82b420716546f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.263594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.338770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a2d3c823a549abbda82b420716546f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.340729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a2d3c823a549abbda82b420716546f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.353168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a2d3c823a549abbda82b420716546f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.362335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:37.369119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a2d3c823a549abbda82b420716546f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.370207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-698a1247-a5a4-4df5-ba96-815ca3af7df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a2d3c823a549abbda82b420716546f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.387819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1964] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:37 2022] DELETE /identity/v3/users/96a2d3c823a549abbda82b420716546f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:37.394278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:37.395679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.410012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e7a58-0fbe-429c-97cd-39cb1ee9a9d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.485518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.486157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.486462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.486729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] PATH_INFO: `/v3/projects/c0d51871d26c4c859a5d0ade43432446` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.487863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0d51871d26c4c859a5d0ade43432446)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.489814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.550256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0d51871d26c4c859a5d0ade43432446', 'request_id': 'req-046ba616-3e00-4a1a-bb39-01d0055f4cad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.558132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:37.567058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0d51871d26c4c859a5d0ade43432446', 'request_id': 'req-046ba616-3e00-4a1a-bb39-01d0055f4cad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.580159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0d51871d26c4c859a5d0ade43432446', 'request_id': 'req-046ba616-3e00-4a1a-bb39-01d0055f4cad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.600834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] The token cache is being invalidate because project c0d51871d26c4c859a5d0ade43432446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:37.601123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.601783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-046ba616-3e00-4a1a-bb39-01d0055f4cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.608066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1965] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:37 2022] DELETE /identity/v3/projects/c0d51871d26c4c859a5d0ade43432446 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:37.610363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.611271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.611730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.641922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:37.645824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.740466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1966] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:37.746830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.747163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.747448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.757191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:37.761763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.778450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e20895-1c8f-409d-b79b-11698fd1628b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.780996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:37.782896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.785392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.786351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.786792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.787986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] PATH_INFO: `/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.790282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bcf375c96e47cab1bc7cd8bf4a3da2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.792874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.800125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1967] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:37.801340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:37.814866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498e73e-131d-42dd-a7c7-8a1adf064bdc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.830631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.831596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.831866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.832161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.833477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:37.833758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.837583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.863560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1968] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:37.864087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:37.872782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bcf375c96e47cab1bc7cd8bf4a3da2', 'request_id': 'req-44c11685-c762-45ba-80b3-00c0944ecec1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.876476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23be16-688b-42af-98e6-11b310f82e5f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:37.883105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bcf375c96e47cab1bc7cd8bf4a3da2', 'request_id': 'req-44c11685-c762-45ba-80b3-00c0944ecec1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.891175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bcf375c96e47cab1bc7cd8bf4a3da2', 'request_id': 'req-44c11685-c762-45ba-80b3-00c0944ecec1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.912640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] The token cache is being invalidate because project e6bcf375c96e47cab1bc7cd8bf4a3da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:37.913499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.915757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-44c11685-c762-45ba-80b3-00c0944ecec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:37.920932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1969] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:37 2022] DELETE /identity/v3/projects/e6bcf375c96e47cab1bc7cd8bf4a3da2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:37.923881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b20a-57e2-4f23-80be-b6345e5120bd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.924567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b20a-57e2-4f23-80be-b6345e5120bd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.924898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b20a-57e2-4f23-80be-b6345e5120bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.951010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:37.951773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.952137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.952495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:37.953191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:37.955253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:37.957336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b89b20a-57e2-4f23-80be-b6345e5120bd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:37.958904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b89b20a-57e2-4f23-80be-b6345e5120bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:37.970421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1970] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:37.978453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:37.979175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:37.979175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.000021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1971] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:37 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:38.003882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.005698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.016832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:38.018037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.129167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f20a-efca-4c3a-9f01-0c24308deb5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.146382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:38.147399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.148802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.150724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.151072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.153000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.156156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.166890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1972] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:38.173178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.189297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b1574-329d-4dd4-90f5-1612245c8b05 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.194189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.194979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1973] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:38.195117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.195341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.195697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.196421 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:38.196724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.197140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.198746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.199607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.215566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1974] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:38.216593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.226764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c826ba9-a565-4257-8b82-e9c1b17d20f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.232152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.232780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.233102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.233377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.234031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.235915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.247605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1975] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:38.248465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.255284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:38.265373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b56d01-29b2-4a6b-acb3-9f5c706fd4ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.270455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.271082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.271361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.271638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.272357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.274628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.280463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce414-a81a-4d12-be65-ef0dc657acee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.284737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.285603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.286401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.286880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.288981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.309922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1976] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:38.314530 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.388948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.391316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.397418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abff7b2-144a-484a-a3a8-e085e1621a6b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.402937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.403569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.403854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.404360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.405159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.406972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1977] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:38.408237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.419181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.426338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022c503-5e80-4ff1-bdbf-be50e0f38f1a service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.436094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.437156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.437624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.438444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.438569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/1978] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:38.439708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.442101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.447033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.461888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6f7f4-6e6d-44a6-a71a-19cd064b82c3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.467420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.468102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.468409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.468681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] PATH_INFO: `/v3/projects/b37fc7dacc11483ca9c6f047d3bd2553/users/b3c011df2b144f758dd286e77875bcc8/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.475349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1979] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/users => generated 294 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:38.480436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.493249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37fc7dacc11483ca9c6f047d3bd2553, user_id=b3c011df2b144f758dd286e77875bcc8, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.495403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd57a64-8bd8-4178-96d3-856e4a11e894 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.496119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f01d65-cd34-4844-8b67-3dbd71051da4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.502361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.503053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.503331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.503604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.504245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.505972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.519078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/1980] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:38 2022] PUT /identity/v3/projects/b37fc7dacc11483ca9c6f047d3bd2553/users/b3c011df2b144f758dd286e77875bcc8/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:38.523724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1981] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:38.524748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.524981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.525271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.525544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.538405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffe882-3856-4779-8b69-38ccd732f94e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.549867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:38.551090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.625129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.626177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.626177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.626384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.627066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.628413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/1982] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:38.628849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.639257 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.641133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1983] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:38.644212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.645405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.671187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9dabee-8797-4aef-bd08-c9d2b84c5b04 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.677991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.684147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.684147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.684147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] PATH_INFO: `/v3/projects/9449d052c6f24c61be6b0264950a3e56/users/6de94710fdcf417b84742d425dc9c862/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.685598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:38.700329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9449d052c6f24c61be6b0264950a3e56, user_id=6de94710fdcf417b84742d425dc9c862, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.704448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4f33b-8f41-4c6a-a974-5f7172edc587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.711694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea1505-7329-4021-80ce-187afd42a36a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.715542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.716148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.716708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.716983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.721343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1984] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:38 2022] PUT /identity/v3/projects/9449d052c6f24c61be6b0264950a3e56/users/6de94710fdcf417b84742d425dc9c862/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:38.733625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.733898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.734163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.740468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.742357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:38.756336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/1985] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:38.764083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] MFA Rules not processed for user `6de94710fdcf417b84742d425dc9c862`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:38.765370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:38.788219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:38.850296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1986] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:38.851021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:38.861659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550406bc-3bbd-49ca-9d67-8f6f481e75ee service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:38.958527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:38.959349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:38.959631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:38.960223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:38.964222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:38.966614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.005838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/1987] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:38 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:18:39.011901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:39.033428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cbd08-5694-495a-9f8c-d29e37964ff3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:39.123049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:39.124014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.124323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.126012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.126753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:39.129378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.161016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1988] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:39 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:39.165032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:39.178985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d558224-4942-4174-8dcd-b284e59b6f38 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:39.184301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:39.184992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.185253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.185518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.187425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:39.187903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.199565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/1989] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:39.203875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:39.217536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc351ef-4ae7-4f5e-9046-8ac1ca77e62b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:39.223554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:39.224241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.224553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.224970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] PATH_INFO: `/v3/projects/32f4548af392412ca28169c33e57eff8/users/861dab7829ec4c59809ec82b84a501b6/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.276796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f4548af392412ca28169c33e57eff8, user_id=861dab7829ec4c59809ec82b84a501b6, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:39.281035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.302693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1990] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:39 2022] PUT /identity/v3/projects/32f4548af392412ca28169c33e57eff8/users/861dab7829ec4c59809ec82b84a501b6/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:39.303999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:39.321847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10498-8e43-4528-841f-b36ecf77cde5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:39.457792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:39.458583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.464790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.465964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.466960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:39.468932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b12e2-91d7-4945-b172-4ee6a233f131 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.484706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/1991] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:39.488613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:39.507541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216efa8e-6f15-48a0-9b5d-c6bcb0eabba2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:39.513784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:39.514737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.515250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.515694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] PATH_INFO: `/v3/projects/32f4548af392412ca28169c33e57eff8/users/861dab7829ec4c59809ec82b84a501b6/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.518513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f4548af392412ca28169c33e57eff8, user_id=861dab7829ec4c59809ec82b84a501b6, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:39.521622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9997a2-2581-49af-b5b5-5cec40cf72fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:39.541965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1992] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:39 2022] PUT /identity/v3/projects/32f4548af392412ca28169c33e57eff8/users/861dab7829ec4c59809ec82b84a501b6/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:39.552096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.553064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.553760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.594304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] MFA Rules not processed for user `861dab7829ec4c59809ec82b84a501b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:39.595681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:39.640333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:39.677285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:39.677565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:39.678293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:39.770975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/1993] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:39.777544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] MFA Rules not processed for user `b3c011df2b144f758dd286e77875bcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:39.778856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:40.000991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1994] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:40.172553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:40.253575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:40.257386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:40.262133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbd78b-5cf3-4771-a12d-f1fd2c5289ec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:40.268184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:40.268882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:40.269175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:40.269520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:40.273922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:40.286627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:40.289018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:40.293649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946c2c-0a5d-429d-9fb9-9aed44368667 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:40.301721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/1995] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:40.301875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:40.302494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:40.302830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:40.303299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:40.318450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:40.320870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a1cdd-ee4d-4793-a484-a607a136adaa service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:40.334825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/1996] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:41.489764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:41.604429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd10b-524e-4b10-9775-0a845ac25816 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:41.611376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:41.612620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:41.613295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:41.613982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] PATH_INFO: `/v3/users/6de94710fdcf417b84742d425dc9c862` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:41.616139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de94710fdcf417b84742d425dc9c862)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:41.618736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:41.704890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de94710fdcf417b84742d425dc9c862'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:41.708306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de94710fdcf417b84742d425dc9c862'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:41.724058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de94710fdcf417b84742d425dc9c862'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:41.779076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:41.781515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de94710fdcf417b84742d425dc9c862'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:41.782676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de94710fdcf417b84742d425dc9c862'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:41.809849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/1997] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:41 2022] DELETE /identity/v3/users/6de94710fdcf417b84742d425dc9c862 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:41.814322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46bcda-c1b7-41cd-a63b-4c03e4f309c5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:41.814822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46bcda-c1b7-41cd-a63b-4c03e4f309c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:41.814822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46bcda-c1b7-41cd-a63b-4c03e4f309c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:41.836987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2a46bcda-c1b7-41cd-a63b-4c03e4f309c5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:41.838532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a46bcda-c1b7-41cd-a63b-4c03e4f309c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:41.934433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/1998] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:41.946427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:41.948106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:42.054523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70278d-88c8-4bd5-a462-574b1f4e0755 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:42.075876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:42.078107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:42.078418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:42.078508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:42.082914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:42.082914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:42.083112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:42.114085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:42.118414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:42.123432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] MFA Rules not processed for user `b3c011df2b144f758dd286e77875bcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:42.124635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:42.139888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/1999] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:42.264679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/2000] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:42.803927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:42.811544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:42.844333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bb3e4-e809-465b-84de-bf5007eafcf8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:42.849365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:42.850007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:42.850290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:42.850564 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] PATH_INFO: `/v3/projects/9449d052c6f24c61be6b0264950a3e56` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:42.851771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9449d052c6f24c61be6b0264950a3e56)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:42.858767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:42.931618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9449d052c6f24c61be6b0264950a3e56', 'request_id': 'req-3cb63da9-e2e4-4db0-af0f-3aa163822660'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:42.938654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9449d052c6f24c61be6b0264950a3e56', 'request_id': 'req-3cb63da9-e2e4-4db0-af0f-3aa163822660'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:42.950205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9449d052c6f24c61be6b0264950a3e56', 'request_id': 'req-3cb63da9-e2e4-4db0-af0f-3aa163822660'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:42.967012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] The token cache is being invalidate because project 9449d052c6f24c61be6b0264950a3e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:42.967302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:42.968741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:42.970816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/2001] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:42 2022] DELETE /identity/v3/projects/9449d052c6f24c61be6b0264950a3e56 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:42.974619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:42.988766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01f11-735c-4762-94d2-335c2acaf646 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:43.061112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:43.062042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:43.062520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:43.062931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] PATH_INFO: `/v3/users/861dab7829ec4c59809ec82b84a501b6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:43.064399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=861dab7829ec4c59809ec82b84a501b6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:43.066841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:43.131214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:43.143120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '861dab7829ec4c59809ec82b84a501b6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.153392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '861dab7829ec4c59809ec82b84a501b6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.165945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '861dab7829ec4c59809ec82b84a501b6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.166877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '861dab7829ec4c59809ec82b84a501b6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.185536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3112079c-5f20-414c-af87-ec4c52ae2449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '861dab7829ec4c59809ec82b84a501b6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.195355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/2002] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:42 2022] DELETE /identity/v3/users/861dab7829ec4c59809ec82b84a501b6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:43.271876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:43.273362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:43.289009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb63da9-e2e4-4db0-af0f-3aa163822660 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:43.369565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:43.370339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:43.370686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:43.371072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] PATH_INFO: `/v3/users/90261602123b4a83a0e417bb2af51d3c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:43.374432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90261602123b4a83a0e417bb2af51d3c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:43.376469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:43.395389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:43.463322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90261602123b4a83a0e417bb2af51d3c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.464215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90261602123b4a83a0e417bb2af51d3c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.487677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90261602123b4a83a0e417bb2af51d3c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.504734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90261602123b4a83a0e417bb2af51d3c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.505688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90261602123b4a83a0e417bb2af51d3c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:43.520439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/2003] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:43 2022] DELETE /identity/v3/users/90261602123b4a83a0e417bb2af51d3c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:43.525301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:43.526011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:43.526327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:43.565328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:43.568552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:43.663624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/2004] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:43.683640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:43.686933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:43.833166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcdc1-dd14-4b73-94c6-f190fa87f61c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:43.851131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:43.852118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:43.853175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:43.853651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:43.875911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:43.878788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cc56b-8b92-4b4c-90f8-e5dbb2662892 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:43.902596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/2005] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:44.077840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.079455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.098600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acce45-0285-4e74-bdfd-108a24d30350 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.110857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.113199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.113828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.114293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] PATH_INFO: `/v3/projects/32f4548af392412ca28169c33e57eff8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.116724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f4548af392412ca28169c33e57eff8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.119619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.203486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f4548af392412ca28169c33e57eff8', 'request_id': 'req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.211352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f4548af392412ca28169c33e57eff8', 'request_id': 'req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.222232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f4548af392412ca28169c33e57eff8', 'request_id': 'req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.238472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] The token cache is being invalidate because project 32f4548af392412ca28169c33e57eff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:44.238663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.240163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.244340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/2006] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:44 2022] DELETE /identity/v3/projects/32f4548af392412ca28169c33e57eff8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:44.247065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78ae4-275f-41db-810d-8518672a9c2f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.247591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78ae4-275f-41db-810d-8518672a9c2f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.248047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78ae4-275f-41db-810d-8518672a9c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.272401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-85d78ae4-275f-41db-810d-8518672a9c2f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:44.274864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-85d78ae4-275f-41db-810d-8518672a9c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.292691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.295600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.346086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:44.360598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/2007] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:44.368416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.368963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.368963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.382786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8fb6a-3c75-4a1d-831f-17eee4fab9a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.387692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.388360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.388770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.389628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] PATH_INFO: `/v3/projects/fcb6c8f78f6446f1a50c383d57df5b82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.389893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:44.390779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb6c8f78f6446f1a50c383d57df5b82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.391044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.392745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.399258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/2008] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:18:44.403386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.412999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634112e-7cd7-4b62-9cd5-3dcb26723594 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.419032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.420088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.420292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.420577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.421983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:44.422108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.423755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.440445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/2009] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:18:44.442121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.450092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb6c8f78f6446f1a50c383d57df5b82', 'request_id': 'req-306a1118-017a-496a-9cb7-5dde62b07c7c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.453372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc4ab-f04a-48c5-9fe1-c31ec05f8899 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.457273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb6c8f78f6446f1a50c383d57df5b82', 'request_id': 'req-306a1118-017a-496a-9cb7-5dde62b07c7c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.466809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:44.470033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb6c8f78f6446f1a50c383d57df5b82', 'request_id': 'req-306a1118-017a-496a-9cb7-5dde62b07c7c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.487762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] The token cache is being invalidate because project fcb6c8f78f6446f1a50c383d57df5b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:44.488049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.489331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.491146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/2010] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:44 2022] DELETE /identity/v3/projects/fcb6c8f78f6446f1a50c383d57df5b82 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:44.495148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.496675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.516871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.517489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.517763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.518032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.518661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.520648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.560598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/2011] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:44 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:44.564677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.566987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.594518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a1118-017a-496a-9cb7-5dde62b07c7c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.688115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b91d8f-bc82-4b1a-be7c-a074bf5c875e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.693819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.694701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.694976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.695244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.696070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.698324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.725731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/2012] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:44 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:44.730646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.736956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.737978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.738351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.738709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] PATH_INFO: `/v3/users/96e6d11694df431482ebd95dd7c11e40` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.739994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e6d11694df431482ebd95dd7c11e40)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.742068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.742542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926e10-4e57-4419-a446-f8add1da5cd7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.747328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.747941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.748218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.748521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.749190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.750750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.764317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/2013] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:44.765926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.771623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:44.776732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43e26f-7d94-4b0e-b081-b0b4178a7720 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.782562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.783260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.783675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.783851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.784521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.786265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.805026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/2014] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:44.806275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:44.818034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999a13-bb7b-4b50-87dd-27212744ed5b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:44.823470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:44.824143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.825893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.826174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] PATH_INFO: `/v3/projects/95480e97a0744286b0630fedbebb0fc2/users/f88ca87438f941a8a596837d004c4188/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.828194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e6d11694df431482ebd95dd7c11e40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.838150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e6d11694df431482ebd95dd7c11e40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.845383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95480e97a0744286b0630fedbebb0fc2, user_id=f88ca87438f941a8a596837d004c4188, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:44.848260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17677018-6a55-4e8b-9993-5f5d252f10d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:44.851928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e6d11694df431482ebd95dd7c11e40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.853580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e6d11694df431482ebd95dd7c11e40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.871477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e6d11694df431482ebd95dd7c11e40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:44.879904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/2015] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:44 2022] PUT /identity/v3/projects/95480e97a0744286b0630fedbebb0fc2/users/f88ca87438f941a8a596837d004c4188/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:44.880805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/2016] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:44 2022] DELETE /identity/v3/users/96e6d11694df431482ebd95dd7c11e40 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:18:44.898104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:44.898388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:44.898642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:44.927393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] MFA Rules not processed for user `f88ca87438f941a8a596837d004c4188`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:44.928625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:44.967400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:45.021476 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/2017] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:45.027548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:45.029278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:45.127430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e242e-af18-4ca3-9054-3b8b856d59fb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:45.131938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:45.132924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:45.133201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:45.133607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:45.144505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:45.146047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:45.156134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/2018] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:18:45.200191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:45.201747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:45.297767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed611fd-51d4-4e2a-9c44-ab2387ebbe8a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:45.302199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:45.302831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:45.303107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:45.303378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] PATH_INFO: `/v3/projects/ccdf68165a454860a88fad3f6489ec69` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:45.304550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccdf68165a454860a88fad3f6489ec69)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:45.308721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:45.365816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccdf68165a454860a88fad3f6489ec69', 'request_id': 'req-60a3975f-7121-4bf3-9638-c6cd7da957fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.372647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccdf68165a454860a88fad3f6489ec69', 'request_id': 'req-60a3975f-7121-4bf3-9638-c6cd7da957fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.379660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccdf68165a454860a88fad3f6489ec69', 'request_id': 'req-60a3975f-7121-4bf3-9638-c6cd7da957fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.397024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] The token cache is being invalidate because project ccdf68165a454860a88fad3f6489ec69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:45.397312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.397962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.400003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/2019] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:45 2022] DELETE /identity/v3/projects/ccdf68165a454860a88fad3f6489ec69 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:45.403717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:45.415434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa561df-a951-4254-9bbc-95e11295d5f0 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:45.484086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:45.484750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:45.485021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:45.485568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] PATH_INFO: `/v3/users/cf71bbeaf0154b46a73bf7c46d3896ef` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:45.488696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf71bbeaf0154b46a73bf7c46d3896ef)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:45.490779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:45.515854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:45.547404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf71bbeaf0154b46a73bf7c46d3896ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.554917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf71bbeaf0154b46a73bf7c46d3896ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.563742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf71bbeaf0154b46a73bf7c46d3896ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.567353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf71bbeaf0154b46a73bf7c46d3896ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.592948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8b3a1d74-f056-40e1-b79c-db6727a8cdff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf71bbeaf0154b46a73bf7c46d3896ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:45.600240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/2020] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:18:45 2022] DELETE /identity/v3/users/cf71bbeaf0154b46a73bf7c46d3896ef => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:46.089008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.091288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:46.114351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3975f-7121-4bf3-9638-c6cd7da957fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.173767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.174428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.174729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.175001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] PATH_INFO: `/v3/projects/ec620c18816f45fa84f24ffcc73745bc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.176121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec620c18816f45fa84f24ffcc73745bc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.178060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.184567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:46.231079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec620c18816f45fa84f24ffcc73745bc', 'request_id': 'req-b60505bb-2733-42f4-8251-ef888729df0b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:46.237656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec620c18816f45fa84f24ffcc73745bc', 'request_id': 'req-b60505bb-2733-42f4-8251-ef888729df0b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:46.245237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec620c18816f45fa84f24ffcc73745bc', 'request_id': 'req-b60505bb-2733-42f4-8251-ef888729df0b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:46.261422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] The token cache is being invalidate because project ec620c18816f45fa84f24ffcc73745bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:18:46.261807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:46.262671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b60505bb-2733-42f4-8251-ef888729df0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:18:46.264987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/2021] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:18:46 2022] DELETE /identity/v3/projects/ec620c18816f45fa84f24ffcc73745bc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:46.271593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.272159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.272427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.296850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:46.298194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:46.377977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2022] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:46.387569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.388123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.388527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.414434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:46.416945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:46.429394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2023] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:46.431101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.445109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d4fc1-1dc9-4185-bad9-404751ecbd19 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.456861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.458144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.458641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.459089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.460406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:18:46.460935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.464419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.487480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2024] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:18:46.496127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.510944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa69d86-fc4c-4f00-b9bb-b8f360531f44 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.519305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.520031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.520542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.520886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.521589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.523607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.571054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2025] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:46.576782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.597960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac2593-08d6-4eaa-8f46-cee9e5bab0e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.668908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.669758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.670031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.670292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.670910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.674248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.699529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2026] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:18:46 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:18:46.704116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.715755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c065-ed94-4fb2-bf75-b3edb895c758 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.722795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.723948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.724082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.724411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.725160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.726918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.741014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2027] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:18:46.746256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.765385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe09b68-72d9-4219-927b-b8ace5738ab7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.777132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.777760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.778032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.778299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.778993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.781453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4378400-1efd-4966-8c2b-2bc5651436d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.801216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2028] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:18:46.801475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:46.824176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44089b-3be2-48ce-8a0a-b435c82738f4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:46.840888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:46.842923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.843872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.846741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] PATH_INFO: `/v3/projects/0df8bd6fe9ca418d9e295aa3f419f5de/users/87689334264c4a939367f091ccfbd642/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.891693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df8bd6fe9ca418d9e295aa3f419f5de, user_id=87689334264c4a939367f091ccfbd642, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:46.894689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:46.942451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2029] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:18:46 2022] PUT /identity/v3/projects/0df8bd6fe9ca418d9e295aa3f419f5de/users/87689334264c4a939367f091ccfbd642/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:18:46.954611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4d41-aae7-4699-ac59-5631fe9dfad5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:46.954972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4d41-aae7-4699-ac59-5631fe9dfad5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:46.955244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec4d41-aae7-4699-ac59-5631fe9dfad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:46.966066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:47.007288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0ec4d41-aae7-4699-ac59-5631fe9dfad5 None None] MFA Rules not processed for user `87689334264c4a939367f091ccfbd642`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:47.009834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0ec4d41-aae7-4699-ac59-5631fe9dfad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:47.163555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2030] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:47.180676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:18:47.183744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:47.265205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a84573-cbf8-4fab-9f5a-300d85e5dc3d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:18:47.269997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:18:47.270630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:47.270905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:47.271175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:47.280622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:18:47.282152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:18:47.293020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2031] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:18:48.744357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6231e3-ebef-48ab-adad-ac36952dbfbe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:18:48.745206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6231e3-ebef-48ab-adad-ac36952dbfbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:18:48.745206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6231e3-ebef-48ab-adad-ac36952dbfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:18:48.774962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd6231e3-ebef-48ab-adad-ac36952dbfbe None None] MFA Rules not processed for user `87689334264c4a939367f091ccfbd642`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:18:48.776169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd6231e3-ebef-48ab-adad-ac36952dbfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:18:48.784574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2032] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:18:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:18:57.288379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:18:58.765933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=34 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:04.204539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:04.209554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:04.336445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef28287-4b34-4ffd-9996-a2afaef458ea service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:04.341731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:04.342385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:04.342656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:04.342920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] PATH_INFO: `/v3/users/ca1143bb11574b94b534e06bdcddf484` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:04.344060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1143bb11574b94b534e06bdcddf484)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:04.346265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:04.405292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1143bb11574b94b534e06bdcddf484'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:04.406351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1143bb11574b94b534e06bdcddf484'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:04.414766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1143bb11574b94b534e06bdcddf484'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:04.427318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1143bb11574b94b534e06bdcddf484'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:04.428026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1143bb11574b94b534e06bdcddf484'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:04.444449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2033] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:04 2022] DELETE /identity/v3/users/ca1143bb11574b94b534e06bdcddf484 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:04.446068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:04.446537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:04.446635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:04.466591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:04.468638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:04.537736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2034] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:04.548333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:04.551224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:04.651951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb2a92-7a25-47ca-b963-8f2a65f3b6fb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:04.655689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:04.656358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:04.656638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:04.656907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:04.668822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:04.670363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:04.681618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2035] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:05.107972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:05.111167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:05.141473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611d557-8418-467b-9b2a-cd07eb3bf410 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:05.148453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:05.149061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:05.149467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:05.149754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] PATH_INFO: `/v3/projects/e278ae8b0fa34455905b9368c5f220e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:05.150927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e278ae8b0fa34455905b9368c5f220e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:05.152886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:05.248678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e278ae8b0fa34455905b9368c5f220e8', 'request_id': 'req-70840351-bbb1-4bc9-87c4-3577bd044259'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.258839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e278ae8b0fa34455905b9368c5f220e8', 'request_id': 'req-70840351-bbb1-4bc9-87c4-3577bd044259'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.274262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e278ae8b0fa34455905b9368c5f220e8', 'request_id': 'req-70840351-bbb1-4bc9-87c4-3577bd044259'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.296805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] The token cache is being invalidate because project e278ae8b0fa34455905b9368c5f220e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:05.297119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.297834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.300527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2036] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:05 2022] DELETE /identity/v3/projects/e278ae8b0fa34455905b9368c5f220e8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:05.304805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:05.327501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8fa70-22e5-4869-97fe-b5754555ef13 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:05.351571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:05.406772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:05.407410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:05.407689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:05.407963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] PATH_INFO: `/v3/users/944d8ed8caa04367b1530f0a783c036b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:05.409308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944d8ed8caa04367b1530f0a783c036b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:05.411235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:05.458909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944d8ed8caa04367b1530f0a783c036b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.459692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944d8ed8caa04367b1530f0a783c036b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.469139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944d8ed8caa04367b1530f0a783c036b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.483924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944d8ed8caa04367b1530f0a783c036b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.484687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3ab1c9e-2f65-48d2-95bd-8f1d08049ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944d8ed8caa04367b1530f0a783c036b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:05.497543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2037] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:05 2022] DELETE /identity/v3/users/944d8ed8caa04367b1530f0a783c036b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:05.875884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:05.878957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:05.895920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840351-bbb1-4bc9-87c4-3577bd044259 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:05.989668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:05.990321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:05.990594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:05.990866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] PATH_INFO: `/v3/projects/d5ac3af3cae44166a153b49ed4aff68c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:05.992739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ac3af3cae44166a153b49ed4aff68c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:05.994735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.031270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:06.056037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ac3af3cae44166a153b49ed4aff68c', 'request_id': 'req-c8db582c-68dd-4367-af2a-6929574e5697'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:06.070385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ac3af3cae44166a153b49ed4aff68c', 'request_id': 'req-c8db582c-68dd-4367-af2a-6929574e5697'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:06.110338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ac3af3cae44166a153b49ed4aff68c', 'request_id': 'req-c8db582c-68dd-4367-af2a-6929574e5697'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:06.127120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] The token cache is being invalidate because project d5ac3af3cae44166a153b49ed4aff68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:06.129358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:06.132655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8db582c-68dd-4367-af2a-6929574e5697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:06.146846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2038] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:05 2022] DELETE /identity/v3/projects/d5ac3af3cae44166a153b49ed4aff68c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:06.147436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.147738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.147993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.181623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:06.182847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:06.292735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2039] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:06.302124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.302414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.302691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.323267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:06.324497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:06.337498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2040] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:06.338706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.351189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93404b10-86ec-47d1-87be-614ee311d1b2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.357067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.358141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.358419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.359076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.359957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:06.360702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.363750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.376685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2041] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:19:06.380838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.392328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7140c8-23f1-4ac5-b729-09ac6e783b21 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.405644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.406305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.406602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.406872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.407507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.409523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.444385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2042] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/projects => generated 399 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:19:06.448464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.465629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fba5f-c96e-406f-bbac-668de1a5a077 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.497618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:06.539760 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.542071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.542473 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.542808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.543660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.545681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.570368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2043] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:06.574719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.585905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee451-aef7-4f73-a8f2-c2eb588a10c9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.591823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.592524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.592798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.593065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.593718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.595483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.607592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2044] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:06.611940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.626034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394941-2462-41cb-878f-a056410b220f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.630840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.631434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.631830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.632107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.633034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.635428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f1eb-60d9-4901-86f6-a361a4b8e41e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.650888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2045] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:06.655251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.666245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da149-b9b8-45f7-a5c3-215bb895413f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.671024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.671654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.671954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.672252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] PATH_INFO: `/v3/projects/9bf33db5a69e4bb3a2031a9b51bf4f96/users/0f10f01218134fcea7524ba6def8bc79/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.686734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf33db5a69e4bb3a2031a9b51bf4f96, user_id=0f10f01218134fcea7524ba6def8bc79, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.689630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.706489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2046] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:06 2022] PUT /identity/v3/projects/9bf33db5a69e4bb3a2031a9b51bf4f96/users/0f10f01218134fcea7524ba6def8bc79/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:06.718014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.718308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.718584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.747608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] MFA Rules not processed for user `0f10f01218134fcea7524ba6def8bc79`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:06.748911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:06.815136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2047] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:06.820007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.888334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d17f0-46a8-42c3-a768-75b50a55001b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:06.893496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:06.894160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:06.894507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:06.894781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:06.895422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:06.897633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:06.930103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2048] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:06.934614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:06.945804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4755e77-e602-4a7e-93e5-0ec9d41a41aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.005616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.006225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.006498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.006751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.007387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.009677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.032975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2049] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:06 2022] POST /identity/v3/users => generated 307 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:07.037081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.047989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc77e6-7140-4d98-b435-17a8cc6df18f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.054070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.054676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.054947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.055212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.055880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.057585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.068443 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2050] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:07.072713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.083569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe354b-63e4-4996-a939-bd8dc11e8fa5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.089653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.090298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.090592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.090848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] PATH_INFO: `/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933/users/e811534099024948b683bd0d98999e54/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.106464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c553b6fad5854b1ea2ecc2fd9be6e933, user_id=e811534099024948b683bd0d98999e54, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.109178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.124121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2051] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:07 2022] PUT /identity/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933/users/e811534099024948b683bd0d98999e54/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:07.128196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.140806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625999-5d3f-4c07-bace-860556a0f396 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.163038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:07.208448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.209052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.209326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.209594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.210231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.211846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681118f1-b99b-4f12-adb6-e1cf0df9ae02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.222012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2052] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:07.226394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.232232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:07.238601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf2741-d939-4d65-aa58-5ba56f29ad57 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.243744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.244428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.244703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.244972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] PATH_INFO: `/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933/users/e811534099024948b683bd0d98999e54/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.246890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c553b6fad5854b1ea2ecc2fd9be6e933, user_id=e811534099024948b683bd0d98999e54, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.249569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.261130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2053] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:07 2022] PUT /identity/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933/users/e811534099024948b683bd0d98999e54/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:07.271188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.271499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.271769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.300063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] MFA Rules not processed for user `e811534099024948b683bd0d98999e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:07.301320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.369724 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2054] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:07.381009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.383335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.454814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8212379-0c05-4c7a-b648-c1d1838e7d4a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.458763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.459361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.459628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.459887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.469795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.471385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.483449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2055] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:07.564866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.566493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.574954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.576403 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.582141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34aa7-50be-4d37-afa2-3bdb76e235d3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.586528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.587150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.587596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.587863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.655557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790dad6-760c-494e-ba8b-19953cb8d04e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:07.660447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:07.661393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.661585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.661957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] PATH_INFO: `/v3/users/f88ca87438f941a8a596837d004c4188` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.663424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88ca87438f941a8a596837d004c4188)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.665377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.680189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:07.680189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c2513-68bd-44da-883f-b374953c9c74 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:07.686822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2056] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:07.739150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88ca87438f941a8a596837d004c4188'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:07.739778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88ca87438f941a8a596837d004c4188'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:07.749785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88ca87438f941a8a596837d004c4188'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:07.773217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88ca87438f941a8a596837d004c4188'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:07.774479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88ca87438f941a8a596837d004c4188'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:07.790985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2057] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:07 2022] DELETE /identity/v3/users/f88ca87438f941a8a596837d004c4188 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:07.795765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:07.796126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:07.796505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:07.818065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:07.819576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.870120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:07.892538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2058] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:07.903034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:07.905013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:07.935994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:08.017042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9481-8d8a-46d9-a744-a2d8f093e039 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.022218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.022763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.023097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.023428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.033623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.035097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c7238-1a05-40d1-bee2-08052fb496e0 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.046731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2059] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:08.323265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.325210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:08.343184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d643f-562c-450a-a3fd-2c3817896b2e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.349671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.350398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.350655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.351042 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] PATH_INFO: `/v3/projects/95480e97a0744286b0630fedbebb0fc2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.352647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95480e97a0744286b0630fedbebb0fc2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.355122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.404608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95480e97a0744286b0630fedbebb0fc2', 'request_id': 'req-34277184-4a76-41e6-82e8-d63f5d6111e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:08.410543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95480e97a0744286b0630fedbebb0fc2', 'request_id': 'req-34277184-4a76-41e6-82e8-d63f5d6111e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:08.419992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95480e97a0744286b0630fedbebb0fc2', 'request_id': 'req-34277184-4a76-41e6-82e8-d63f5d6111e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:08.432124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] The token cache is being invalidate because project 95480e97a0744286b0630fedbebb0fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:08.432440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:08.433103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34277184-4a76-41e6-82e8-d63f5d6111e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:08.434614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2060] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:08 2022] DELETE /identity/v3/projects/95480e97a0744286b0630fedbebb0fc2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:08.439893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.440404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.440404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.461909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:08.463865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:08.529503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2061] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:08.537659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.537947 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.538218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.559331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:08.560589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:08.569981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2062] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:08.574626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.584926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718b11c-60a2-431b-bb8c-aad1d43b580f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.590420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.591155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.591435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.591706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.592402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:08.592716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.594353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.609623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2063] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:19:08.610791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.620328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d866d0-82c9-47fb-acb9-56c26db03ee2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.625316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.625945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.626210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.626472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.627178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.629218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.659428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2064] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:08 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:19:08.663733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.673853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c930-e74b-40ca-8450-84908a2580a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.748099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.748099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.748462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.748706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.749358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.751694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.781522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2065] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:08 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:08.783956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.795159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf5a8f-2df9-406a-80f7-05819a9d6204 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.799795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.800464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.800737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.800999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.801639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.803430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.823200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2066] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:08.824457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.835992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311503-42f8-4783-8186-97e9ea20aa67 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.840669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.841290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.841569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.841840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.842485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.844174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9be7-dbcc-4f87-9c30-87a17f244cf0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.862119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2067] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:08.863483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:08.877310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a65a-0f41-4418-bddb-c473e4c65668 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:08.883839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:08.883839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.883839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.883839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] PATH_INFO: `/v3/projects/887526df7939466fbf0bbff2e29fb861/users/2db8233bc6204457942a6b7343049a01/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.902466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=887526df7939466fbf0bbff2e29fb861, user_id=2db8233bc6204457942a6b7343049a01, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:08.905899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:08.937625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2068] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:08 2022] PUT /identity/v3/projects/887526df7939466fbf0bbff2e29fb861/users/2db8233bc6204457942a6b7343049a01/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:08.939931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db94383-3c78-4d50-bf28-a2286092c4ff None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:08.940351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db94383-3c78-4d50-bf28-a2286092c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:08.940635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db94383-3c78-4d50-bf28-a2286092c4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:08.974654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8db94383-3c78-4d50-bf28-a2286092c4ff None None] MFA Rules not processed for user `2db8233bc6204457942a6b7343049a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:08.975989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8db94383-3c78-4d50-bf28-a2286092c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:08.983971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:09.049966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2069] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:09.059222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:09.060862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:09.126091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4496fe1-1da3-4cfd-9d19-21f6efdb2bfb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:09.130382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:09.131068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:09.131405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:09.131731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:09.143079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:09.144747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:09.156957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2070] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:11.262487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:11.263023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:11.263023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:11.285588 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] MFA Rules not processed for user `b3c011df2b144f758dd286e77875bcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:11.287210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:11.367730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2071] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:11.946814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:11.957190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:12.118420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda326-fc94-45b6-8a31-94653b5fbca7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:12.128767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:12.131402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.131402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.131402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] PATH_INFO: `/v3/users/2db8233bc6204457942a6b7343049a01` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.132298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db8233bc6204457942a6b7343049a01)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:12.134275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:12.156178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:12.157662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.191072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b0fb-9516-4163-ae82-4765665e88e4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:12.203859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:12.204878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.205245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.207115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] PATH_INFO: `/v3/users/0f10f01218134fcea7524ba6def8bc79` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.213491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f10f01218134fcea7524ba6def8bc79)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:12.215424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:12.262434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db8233bc6204457942a6b7343049a01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.280642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db8233bc6204457942a6b7343049a01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.294376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db8233bc6204457942a6b7343049a01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.295601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db8233bc6204457942a6b7343049a01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.307091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f10f01218134fcea7524ba6def8bc79'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.309036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f10f01218134fcea7524ba6def8bc79'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.318370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9ec3c9ec-e453-4901-b887-3605cb11fd38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db8233bc6204457942a6b7343049a01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.325660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f10f01218134fcea7524ba6def8bc79'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.330003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2072] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:11 2022] DELETE /identity/v3/users/2db8233bc6204457942a6b7343049a01 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:12.340045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.340490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.340755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.348481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f10f01218134fcea7524ba6def8bc79'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.350451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de55298d-abdc-4287-94e9-3141212c7658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f10f01218134fcea7524ba6def8bc79'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:12.374455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:12.376994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2073] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:12 2022] DELETE /identity/v3/users/0f10f01218134fcea7524ba6def8bc79 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:12.378528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.382776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.383386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.387366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.453972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:12.457948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.513433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:12.518439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2074] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:12.526028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:12.527828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.579279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:12.591020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2075] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:12.617582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388461ce-cdf7-40cb-a5b4-823a857b8fea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:12.621894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:12.622265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.622604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.622937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.633799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:12.636122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dadcc-2f04-4414-9597-e65de2149c8c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:12.648663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:12.648663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.663151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c1f45-7956-4f96-8f81-216c3e231928 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:12.664447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2076] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:19:12.668231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:12.670381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.670843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.671169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.684019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:12.685617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29921dbb-438b-4bc5-a99f-9e903164ada7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:12.704796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2077] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:12.735555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.736915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.739115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.742328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2078] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:19:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:19:12.830542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:12.831197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:12.831575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:12.855857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:12.857751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:12.989128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:12.989561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2079] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:12.991362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.016810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.017734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ca72d-7629-4da1-b3f6-6331ed1c67c3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.020181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.029500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.031345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.031633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.031967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] PATH_INFO: `/v3/users/69c8a0130c8e42c5b4a6ffe43d15865e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.036589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c8a0130c8e42c5b4a6ffe43d15865e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.038577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.043225 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0726670-26c7-4474-b9a3-b4ae89d14c5c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.052971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.054416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.054699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.055068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.070499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.073680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.099542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2080] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:13.101041 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.103984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.125886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88e8c-d05d-49f0-a78f-0ab66faf2fe3 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.132861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.135221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.135404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.135622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c8a0130c8e42c5b4a6ffe43d15865e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.135873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] PATH_INFO: `/v3/projects/887526df7939466fbf0bbff2e29fb861` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.138360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] RBAC: Authorizing `identity:delete_project(project_id=887526df7939466fbf0bbff2e29fb861)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.141750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.150534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c8a0130c8e42c5b4a6ffe43d15865e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.179485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c8a0130c8e42c5b4a6ffe43d15865e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.180831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c8a0130c8e42c5b4a6ffe43d15865e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.200575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c8a0130c8e42c5b4a6ffe43d15865e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.207214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2081] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:12 2022] DELETE /identity/v3/users/69c8a0130c8e42c5b4a6ffe43d15865e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:13.208416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.209950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.214362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887526df7939466fbf0bbff2e29fb861', 'request_id': 'req-6c931f2b-5541-497c-856e-41360635520a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.221353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887526df7939466fbf0bbff2e29fb861', 'request_id': 'req-6c931f2b-5541-497c-856e-41360635520a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.229684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887526df7939466fbf0bbff2e29fb861', 'request_id': 'req-6c931f2b-5541-497c-856e-41360635520a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.262319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] The token cache is being invalidate because project 887526df7939466fbf0bbff2e29fb861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:13.266261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.268030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c931f2b-5541-497c-856e-41360635520a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.270086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2082] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:13 2022] DELETE /identity/v3/projects/887526df7939466fbf0bbff2e29fb861 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:13.276736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee559b-566e-44f9-b328-b251d48054cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.277455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee559b-566e-44f9-b328-b251d48054cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.277891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee559b-566e-44f9-b328-b251d48054cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.297449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71d935-79e7-4eff-a03a-a76c60501471 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.303393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bee559b-566e-44f9-b328-b251d48054cd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:13.304423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.304764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bee559b-566e-44f9-b328-b251d48054cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.305418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.305766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.306240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] PATH_INFO: `/v3/projects/9bf33db5a69e4bb3a2031a9b51bf4f96` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.308057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf33db5a69e4bb3a2031a9b51bf4f96)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.310695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.319646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2083] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:13.327312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.328110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.328110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.373179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:13.374443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.388633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:13.393027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:13.394558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf33db5a69e4bb3a2031a9b51bf4f96', 'request_id': 'req-a9178da2-8b29-4890-9742-a81ce7028e6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.408887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf33db5a69e4bb3a2031a9b51bf4f96', 'request_id': 'req-a9178da2-8b29-4890-9742-a81ce7028e6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.424040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf33db5a69e4bb3a2031a9b51bf4f96', 'request_id': 'req-a9178da2-8b29-4890-9742-a81ce7028e6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.441632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] The token cache is being invalidate because project 9bf33db5a69e4bb3a2031a9b51bf4f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:13.442078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.443002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.444902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2084] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:13 2022] DELETE /identity/v3/projects/9bf33db5a69e4bb3a2031a9b51bf4f96 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:13.449154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.451188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.476915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2085] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:13.481405 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.493427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450c15c-c3f1-4599-9d3b-808151ba6c00 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.500167 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.501228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.503023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.503444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.504504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:13.505269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.507823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.534118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2086] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:19:13.535260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.537128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.554420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c58625-5a5f-429b-8fd9-c0121de061c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.560450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.561649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.561649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.561879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] PATH_INFO: `/v3/projects/8459b14a54824536b415b2b7336389df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.563052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8459b14a54824536b415b2b7336389df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.565157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.583507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178da2-8b29-4890-9742-a81ce7028e6a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.592004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.592866 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.593238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.593605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] PATH_INFO: `/v3/users/e811534099024948b683bd0d98999e54` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.595050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e811534099024948b683bd0d98999e54)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.599777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.632380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8459b14a54824536b415b2b7336389df', 'request_id': 'req-ef022b9e-d7db-4698-acd8-b5747c5cc286'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.640973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8459b14a54824536b415b2b7336389df', 'request_id': 'req-ef022b9e-d7db-4698-acd8-b5747c5cc286'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.652623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8459b14a54824536b415b2b7336389df', 'request_id': 'req-ef022b9e-d7db-4698-acd8-b5747c5cc286'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.679835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] The token cache is being invalidate because project 8459b14a54824536b415b2b7336389df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:13.680197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.686354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e811534099024948b683bd0d98999e54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.686524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.696572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e811534099024948b683bd0d98999e54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.697067 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2087] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:13 2022] DELETE /identity/v3/projects/8459b14a54824536b415b2b7336389df => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:13.697975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:13.699577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.716083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e811534099024948b683bd0d98999e54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.716952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e811534099024948b683bd0d98999e54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.738413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea5d9318-6731-417e-8646-5aa9955f1182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e811534099024948b683bd0d98999e54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:13.751135 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2088] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:13 2022] DELETE /identity/v3/users/e811534099024948b683bd0d98999e54 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:13.753863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef9e1-90bb-47a4-a1ae-16fdfcb1b508 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.754559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef9e1-90bb-47a4-a1ae-16fdfcb1b508 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.754949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef9e1-90bb-47a4-a1ae-16fdfcb1b508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.807162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19cef9e1-90bb-47a4-a1ae-16fdfcb1b508 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:13.808478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19cef9e1-90bb-47a4-a1ae-16fdfcb1b508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:13.822277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef022b9e-d7db-4698-acd8-b5747c5cc286 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:13.887571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:13.943835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:13.950463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:13.951104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.951385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.951661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:13.952315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:13.954347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:13.975581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2089] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:13.984651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:13.985751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:13.986009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.013108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2090] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:13 2022] POST /identity/v3/projects => generated 373 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:14.013318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:14.014434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.016698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.018349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.116426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2091] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:14.118835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.148410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebadcd-1ae3-4fa5-a269-7b11e210cff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.148594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65b5ed-a3ac-4ccc-b066-73bb9aa927aa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.155396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.157214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.157423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.157709 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.158249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.158534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.159117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.159676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.160420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:14.160638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.160937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.163058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.163937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.184432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2092] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:19:14.188226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.193202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2093] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:14.196987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.208090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17140ad5-bcf3-4f99-b345-01ec3f735c93 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.211164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c47007-a23a-49d4-91ff-2e3bb961cb91 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.214787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.216118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.216487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.216863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.218091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.218210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.218842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.219187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.219516 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.220289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.222130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.222273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.241527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2094] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:14.245813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.258711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2095] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/projects => generated 391 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:19:14.264033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70018d-069e-431e-8c5e-50416e0a2f21 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.264413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.331954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.333409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.333901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.334341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.335149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.336939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.351913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2096] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:19:14.351913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.351913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.364695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3bf88-9537-4563-be7f-aa0f91d0031c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.364898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214670-0557-43a8-b535-2672fddd5ced admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.369625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.369801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.370241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.370435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.370533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.370760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.370864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.371213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] PATH_INFO: `/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.371379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.372498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c553b6fad5854b1ea2ecc2fd9be6e933)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.373446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.374526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.405006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2097] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:14.408996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.410000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:14.420888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbc099-c23a-45af-97df-84926adff65e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.430371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c553b6fad5854b1ea2ecc2fd9be6e933', 'request_id': 'req-13b79a24-4183-4098-be0a-271a993b3658'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:14.437852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c553b6fad5854b1ea2ecc2fd9be6e933', 'request_id': 'req-13b79a24-4183-4098-be0a-271a993b3658'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:14.445907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c553b6fad5854b1ea2ecc2fd9be6e933', 'request_id': 'req-13b79a24-4183-4098-be0a-271a993b3658'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:14.464555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] The token cache is being invalidate because project c553b6fad5854b1ea2ecc2fd9be6e933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:14.464971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:14.465658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:14.467396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2098] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:14 2022] DELETE /identity/v3/projects/c553b6fad5854b1ea2ecc2fd9be6e933 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:14.468187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.469651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.477844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:14.496051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.496810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.497090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.497362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] PATH_INFO: `/v3/projects/1bcc88b363f4436eaf8e374707268930/users/fb3d0eeeb27d46279829bb2f028d5d8d/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.516632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bcc88b363f4436eaf8e374707268930, user_id=fb3d0eeeb27d46279829bb2f028d5d8d, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.520211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd93c9-fb75-46cc-88c0-ef9dac8002b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.541924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b79a24-4183-4098-be0a-271a993b3658 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.544364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2099] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:14 2022] PUT /identity/v3/projects/1bcc88b363f4436eaf8e374707268930/users/fb3d0eeeb27d46279829bb2f028d5d8d/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:14.548289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.549068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.549498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.572021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:14.573316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.607254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.608482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.608957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.609485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.610675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.612602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40131-0c28-401f-b394-e574de416ca9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.628632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2100] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:14.628632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.628632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.628632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.641848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2101] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:14.642779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.653519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68828c-3b55-4f48-a8a8-1dbba18ac0ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.658543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.659347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.659592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.659862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.660572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.662563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a893c-dc82-42df-b015-d9af616980d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.663839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] MFA Rules not processed for user `fb3d0eeeb27d46279829bb2f028d5d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:14.665467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.674063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2102] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:14.675475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.675760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.676028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.697792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:14.699014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.712804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2103] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:14.713960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.730028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:14.730484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975822-67f3-4304-a094-29ce263a0f67 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.742132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.743178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.743714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.744152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] PATH_INFO: `/v3/projects/45bb3a603ddc4d3b8042b6ae687c896e/users/3a1cd35a257f4b0f99141580c14bad48/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.758973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2104] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:14.760198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.770824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bb3a603ddc4d3b8042b6ae687c896e, user_id=3a1cd35a257f4b0f99141580c14bad48, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.775401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.777326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1643eb-9234-48ea-a6bc-a5c037219a92 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.783941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.785077 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.785324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.785594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.786243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:14.786592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.788402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c43c5f-c91b-4a14-8df3-b6f659e1dc1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.798393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2105] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:14 2022] PUT /identity/v3/projects/45bb3a603ddc4d3b8042b6ae687c896e/users/3a1cd35a257f4b0f99141580c14bad48/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:14.799887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.801944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2106] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:19:14.803838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.804133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.804431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.804665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.823239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:14.824504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.872578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cded2f-80d7-44a1-80c0-5d115548e372 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.877038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.877888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.878234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.878459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.891840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2107] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:14.892736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.903205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d72db-85c8-4e4f-9978-5fd5bc42f61f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:14.908004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:14.908649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:14.908957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:14.909228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:14.909863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.912002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.948700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:14.949305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2108] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:14.950568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.952678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:14.968157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:14.969807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:14.980157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2109] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:14.981123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:14.994006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a2673-9d8c-479d-b0a5-d1c84101ecd0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.025435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c0691-ea3e-4a18-845e-ee189e6dc983 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.030611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.031391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.031758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.032105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.064585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.065944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.066393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.066860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.067653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.069878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.094374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2110] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:14 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:15.095381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.097224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:15.102753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.104281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.110158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bcb5-8956-4229-8d36-516abee610b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.115094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2111] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:15.115987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.116122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.117164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.117429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.117747 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] PATH_INFO: `/v3/users/fb3d0eeeb27d46279829bb2f028d5d8d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.118912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3d0eeeb27d46279829bb2f028d5d8d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.120897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.121195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:15.129118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948ff96-2855-418e-99e4-edc9cc75a755 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.135786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.137283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.137706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.138433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.139355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.141610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.161743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2112] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:15.163061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.179028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2000fc-2446-4d6b-959a-d1713799d3bd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.184977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.186179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.186652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.187069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.187979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.189968 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.206182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2113] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:19:15.209885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.211059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3d0eeeb27d46279829bb2f028d5d8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:15.220788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f4b4c-bf1e-4550-82e4-dd2f0504f5e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.228041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3d0eeeb27d46279829bb2f028d5d8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:15.228616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.231876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.232166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.232475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] PATH_INFO: `/v3/projects/d6daf1c34b484f96bb981a8e2c2c5206/users/2ca5ea8e167b44dc9b43c987f4aaff35/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.240141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3d0eeeb27d46279829bb2f028d5d8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:15.242333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3d0eeeb27d46279829bb2f028d5d8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:15.256799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6daf1c34b484f96bb981a8e2c2c5206, user_id=2ca5ea8e167b44dc9b43c987f4aaff35, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.259560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09644a-4e8b-4644-b424-a7815e92025c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.282786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d3e444a9-72ed-4191-b66d-e9e08283adaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3d0eeeb27d46279829bb2f028d5d8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:15.285990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2114] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:15 2022] PUT /identity/v3/projects/d6daf1c34b484f96bb981a8e2c2c5206/users/2ca5ea8e167b44dc9b43c987f4aaff35/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:15.297040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.297356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.297688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.301988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2115] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:15 2022] DELETE /identity/v3/users/fb3d0eeeb27d46279829bb2f028d5d8d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:15.304731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.305358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.305636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.331633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:15.333020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:15.333435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] MFA Rules not processed for user `2ca5ea8e167b44dc9b43c987f4aaff35`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:15.335233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:15.393085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:15.416078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2116] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:15.420809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.422281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:15.427710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2117] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:15.437893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.438389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4375da6-baaf-44da-a72b-93f17c862ba7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.440695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:15.445469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.446084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.446361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.446631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.447294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.449392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.492911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2118] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:15 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:15.494000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.543148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1be170-b2b2-449d-9280-b31ee0821afe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.592845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6e2f6-20af-4bb6-8e97-0662109a0984 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.599971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.600739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.601419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.601774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.602547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.605972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.610695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.611323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.612087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.612232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.639650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2119] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:15 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:15.643819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.644245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.646864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.658494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43b360-7748-4bc7-83e0-6f59b92af5a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.661580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2120] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:15.664592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.665392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.665840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.669330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.669330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.669330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.680780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2121] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:15.684837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.697729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e985f-1b41-429e-b1b5-cea2c41b2ade service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.706894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.707977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.708737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.709204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] PATH_INFO: `/v3/projects/db9151cf06ee4e9e94a8cd1619827e85/users/086b104d0c9a479db1b066b63f39b24a/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.736021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9151cf06ee4e9e94a8cd1619827e85, user_id=086b104d0c9a479db1b066b63f39b24a, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.739851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.757243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2122] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:15 2022] PUT /identity/v3/projects/db9151cf06ee4e9e94a8cd1619827e85/users/086b104d0c9a479db1b066b63f39b24a/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:15.763361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.770117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:15.776716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf61c8-aa62-46b2-90d3-5c89549403ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.857984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.859383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.859717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.860061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.860907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.863194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb0c5-0204-4c30-b3d8-911102e73ee4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.882858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:15.885423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2123] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:15.902038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7fc92-2e10-496d-90ef-a9ba1146abf4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:15.926680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:15.927286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.927558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.927825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] PATH_INFO: `/v3/projects/db9151cf06ee4e9e94a8cd1619827e85/users/086b104d0c9a479db1b066b63f39b24a/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.930166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9151cf06ee4e9e94a8cd1619827e85, user_id=086b104d0c9a479db1b066b63f39b24a, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:15.935620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:15.961813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:15.962060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:15.962337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:15.965250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2124] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:15 2022] PUT /identity/v3/projects/db9151cf06ee4e9e94a8cd1619827e85/users/086b104d0c9a479db1b066b63f39b24a/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:16.004307 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] MFA Rules not processed for user `086b104d0c9a479db1b066b63f39b24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:16.005995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:16.114703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2125] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:16.129531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:16.133065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:16.212479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351530bb-713b-43b8-8093-32b55175c008 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:16.218885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:16.219620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:16.219967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:16.220399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:16.233195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:16.235070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:16.247334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2126] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:16.390785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:16.403176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024881e-c7f0-4163-99aa-8a7b0d5739a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:16.413121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:16.413841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:16.414118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:16.414430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:16.446986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:16.448468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:16.474358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:16.564400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:16.566140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:16.577892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2127] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:16.598041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756b8d3-9230-4394-9235-5e913f7d01de service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:16.602765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:16.603543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:16.604641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:16.604641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] PATH_INFO: `/v3/projects/1bcc88b363f4436eaf8e374707268930` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:16.605370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bcc88b363f4436eaf8e374707268930)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:16.607309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:16.688194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bcc88b363f4436eaf8e374707268930', 'request_id': 'req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:16.696722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bcc88b363f4436eaf8e374707268930', 'request_id': 'req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:16.713753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bcc88b363f4436eaf8e374707268930', 'request_id': 'req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:16.731698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] The token cache is being invalidate because project 1bcc88b363f4436eaf8e374707268930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:16.731698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:16.732835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70fb3f59-7ba5-457d-961d-f2de62ab5fe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:16.735104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2128] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:16 2022] DELETE /identity/v3/projects/1bcc88b363f4436eaf8e374707268930 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:19.456963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:19.494251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea749-d6ff-4dfe-969b-33f2d418084f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:19.535582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:19.628563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:19.629719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:19.630148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:19.630577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] PATH_INFO: `/v3/users/2ca5ea8e167b44dc9b43c987f4aaff35` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:19.633169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca5ea8e167b44dc9b43c987f4aaff35)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:19.640863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:19.761400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca5ea8e167b44dc9b43c987f4aaff35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:19.763275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca5ea8e167b44dc9b43c987f4aaff35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:19.779406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca5ea8e167b44dc9b43c987f4aaff35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:19.797437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca5ea8e167b44dc9b43c987f4aaff35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:19.798562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca5ea8e167b44dc9b43c987f4aaff35'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:19.814518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2129] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:19 2022] DELETE /identity/v3/users/2ca5ea8e167b44dc9b43c987f4aaff35 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:19.819161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08441-2bb2-40f3-925f-954392f95c1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:19.819644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08441-2bb2-40f3-925f-954392f95c1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:19.819961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd08441-2bb2-40f3-925f-954392f95c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:19.853505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6fd08441-2bb2-40f3-925f-954392f95c1e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:19.855125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6fd08441-2bb2-40f3-925f-954392f95c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:19.967016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2130] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:20.001880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:20.003802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:20.215242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c585c5-7612-440f-8127-342bf72c5845 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:20.229190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:20.229843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:20.230133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:20.230392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:20.248070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:20.250583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:20.264598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2131] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:20.758114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4e68-15c4-40ba-be6c-7c323627333b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:20.758408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4e68-15c4-40ba-be6c-7c323627333b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:20.758663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4e68-15c4-40ba-be6c-7c323627333b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:20.791445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc8a4e68-15c4-40ba-be6c-7c323627333b None None] MFA Rules not processed for user `3a1cd35a257f4b0f99141580c14bad48`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:20.793066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc8a4e68-15c4-40ba-be6c-7c323627333b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:20.828871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:20.856642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:20.859539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:20.875496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a356ff6-f1ec-43a0-869b-3b61ae0b8422 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:20.880247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2132] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:20.882218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:20.882867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:20.883139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:20.883404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] PATH_INFO: `/v3/projects/d6daf1c34b484f96bb981a8e2c2c5206` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:20.884571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6daf1c34b484f96bb981a8e2c2c5206)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:20.886855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:20.891021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:20.891292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:20.891552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:20.920469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] MFA Rules not processed for user `3a1cd35a257f4b0f99141580c14bad48`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:20.921700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:20.929676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2133] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:20.941636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:20.943043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:20.951618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:20.962042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6daf1c34b484f96bb981a8e2c2c5206', 'request_id': 'req-9620f340-4365-439a-a7e7-9c2579c03468'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:20.972997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6daf1c34b484f96bb981a8e2c2c5206', 'request_id': 'req-9620f340-4365-439a-a7e7-9c2579c03468'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:20.980433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6daf1c34b484f96bb981a8e2c2c5206', 'request_id': 'req-9620f340-4365-439a-a7e7-9c2579c03468'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:20.999492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] The token cache is being invalidate because project d6daf1c34b484f96bb981a8e2c2c5206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:20.999748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.000552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.002240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2134] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:20 2022] DELETE /identity/v3/projects/d6daf1c34b484f96bb981a8e2c2c5206 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:21.006447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:21.007866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:21.032020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56115-3f94-430b-bcca-a1727fddab12 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:21.098399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620f340-4365-439a-a7e7-9c2579c03468 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:21.104039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:21.104721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:21.104995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:21.105272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] PATH_INFO: `/v3/users/086b104d0c9a479db1b066b63f39b24a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:21.105885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:21.106424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086b104d0c9a479db1b066b63f39b24a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:21.106591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:21.106864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:21.107131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:21.108381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:21.108983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:21.172230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086b104d0c9a479db1b066b63f39b24a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.172910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086b104d0c9a479db1b066b63f39b24a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.178635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:21.180329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa05a2-7cdf-4313-850d-e0cce3f8d6ae service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:21.183010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086b104d0c9a479db1b066b63f39b24a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.184219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:21.192463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2135] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:21.198764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086b104d0c9a479db1b066b63f39b24a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.199491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086b104d0c9a479db1b066b63f39b24a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:21.213040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2136] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:21 2022] DELETE /identity/v3/users/086b104d0c9a479db1b066b63f39b24a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:21.575675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:21.575975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:21.576246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:21.620411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:21.622420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:21.753588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2137] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:19:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:21.755575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:21.757164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:21.829108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:21.832142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:21.883791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:21.919520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3af8b1-82c3-4f61-ac3f-8b87600f61b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:21.935444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:21.937988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:21.937988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:21.937988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] PATH_INFO: `/v3/projects/db9151cf06ee4e9e94a8cd1619827e85` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:21.948171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9151cf06ee4e9e94a8cd1619827e85)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:21.948171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.013004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904584-c8eb-4bbc-b9d7-e1bb66b6e7bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.036492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.037071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.037349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.037613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.053540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.056089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1c36d-aa35-4fab-ba3a-2315b07ffbe6 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.071435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2138] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:19:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 11:19:22.076824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9151cf06ee4e9e94a8cd1619827e85', 'request_id': 'req-6a128608-fa09-4d6e-add5-62f91083c811'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:22.094780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9151cf06ee4e9e94a8cd1619827e85', 'request_id': 'req-6a128608-fa09-4d6e-add5-62f91083c811'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:22.106278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9151cf06ee4e9e94a8cd1619827e85', 'request_id': 'req-6a128608-fa09-4d6e-add5-62f91083c811'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:22.140125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] The token cache is being invalidate because project db9151cf06ee4e9e94a8cd1619827e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:22.142704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:22.147753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6a128608-fa09-4d6e-add5-62f91083c811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:22.152362 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2139] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:21 2022] DELETE /identity/v3/projects/db9151cf06ee4e9e94a8cd1619827e85 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:22.156850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.157392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.158209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.184560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:22.185891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:22.324434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2140] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:22.333664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef5f0-21d2-4a3a-8f22-045ce2fe478e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.334667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef5f0-21d2-4a3a-8f22-045ce2fe478e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.336861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef5f0-21d2-4a3a-8f22-045ce2fe478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.409099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-397ef5f0-21d2-4a3a-8f22-045ce2fe478e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:22.413353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-397ef5f0-21d2-4a3a-8f22-045ce2fe478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:22.448352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2141] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:22.451919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.462009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.462340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.463949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.470839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a48ed-484c-4832-8eb4-9c5801f063a6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.489749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.490511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.490785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.491052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.491720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:22.492017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.493710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.512787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] MFA Rules not processed for user `3a1cd35a257f4b0f99141580c14bad48`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:22.514018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:22.524498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2142] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:19:22.529850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.553103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6da2b1-50c6-44ae-91d0-03e039b117aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.570616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.571590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.571920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.572468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.573615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.577045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.612088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2143] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:22.616835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:22.631641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2144] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:22 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:22.650557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.682573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97779816-3f8c-484b-906b-0ef80f2d8373 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.768683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.770245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.770567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.770895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.771881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.774644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.814790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2145] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:22 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:22.820039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.836093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ca562-c6e4-47ad-9a3a-a1d1bbf7826a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.846289 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.847296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.848742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.849063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.849920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.852551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.883763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2146] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:19:22.887834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.898710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:22.904070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12767-827d-4490-8744-4ded7dd88de1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.913386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.914065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.914453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.915193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.916918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.918761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123b5ba-94fa-450a-8b05-161651248bae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:22.932391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2147] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:22.936876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:22.947895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ee049-c28e-4637-978a-5884cfba2fdb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:22.957237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:22.959334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:22.960158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:22.962368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] PATH_INFO: `/v3/projects/5c4d567e8a2948afa9c76331c70493e9/users/81acf525c1254fe9b8ce123034a3841f/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:22.989682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4d567e8a2948afa9c76331c70493e9, user_id=81acf525c1254fe9b8ce123034a3841f, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:22.992470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e6113-6901-4945-a87c-c066805d1a29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:23.015125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2148] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:22 2022] PUT /identity/v3/projects/5c4d567e8a2948afa9c76331c70493e9/users/81acf525c1254fe9b8ce123034a3841f/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:23.027276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.027579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.027850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.070744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] MFA Rules not processed for user `81acf525c1254fe9b8ce123034a3841f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:23.072380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:23.176950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.177292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.177561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.191620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2149] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:23.200368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:23.201592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:23.202763 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:23.204222 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:23.269172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2f4ed-94ff-4034-9436-1981b0640b4d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:23.273010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:23.273948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.274283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.274555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.278114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2150] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:23.285999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:23.289817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77056a86-7214-4743-871f-dbc65d4c1b8c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:23.291828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:23.293358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:23.312850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2151] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:23.376586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c839de-196e-45ca-bf92-8acddd724523 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:23.382568 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:23.383328 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.383695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.384052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.393689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:23.395662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:23.408361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2152] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:19:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 11:19:23.759034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec754-f358-4fbd-b903-5ae10ef572d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.759034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec754-f358-4fbd-b903-5ae10ef572d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.759268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec754-f358-4fbd-b903-5ae10ef572d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.786925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9beec754-f358-4fbd-b903-5ae10ef572d6 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:23.788456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9beec754-f358-4fbd-b903-5ae10ef572d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:23.804447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2153] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:23.823235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:23.834528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538624-2642-4941-a9dc-50d10381dd99 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:23.842723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:23.844591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:23.844963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:23.846263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:23.857495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:23.860592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:23.875043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2154] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:19:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:19:24.343121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:24.343592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:24.343840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:24.365843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:24.367330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:24.376983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2155] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:19:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:24.393059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:24.404490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26498-8ad9-470e-9cd5-e9302e9e0d1d service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:24.409851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:24.410459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:24.410731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:24.410996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:24.421302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:24.423064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:24.432721 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2156] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:19:27.294730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:27.372419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d437a00-9565-48a4-a750-c63ac8fb18e7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:27.378152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:27.379080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.380044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.380502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] PATH_INFO: `/v3/users/81acf525c1254fe9b8ce123034a3841f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.381927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81acf525c1254fe9b8ce123034a3841f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:27.384049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:27.409589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:27.437417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:27.440591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:27.456897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81acf525c1254fe9b8ce123034a3841f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.458344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81acf525c1254fe9b8ce123034a3841f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.461810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc98-b246-4220-8bd8-252e25d69e90 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:27.467807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:27.468880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.469406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.469806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] PATH_INFO: `/v3/users/87689334264c4a939367f091ccfbd642` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.471096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87689334264c4a939367f091ccfbd642)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:27.473187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:27.475643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81acf525c1254fe9b8ce123034a3841f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.490893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81acf525c1254fe9b8ce123034a3841f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.493047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-225efa6a-9a4e-4c78-87df-cd68cdf93436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81acf525c1254fe9b8ce123034a3841f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.517006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2157] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:27 2022] DELETE /identity/v3/users/81acf525c1254fe9b8ce123034a3841f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:27.519921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.520347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.520872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.527365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:27.548046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:27.550211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:27.561999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87689334264c4a939367f091ccfbd642'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.576196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87689334264c4a939367f091ccfbd642'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.589433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87689334264c4a939367f091ccfbd642'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.591314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87689334264c4a939367f091ccfbd642'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.610787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724acae7-aef8-496c-934c-53e9cd2bab64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87689334264c4a939367f091ccfbd642'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:27.621940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2158] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:19:27 2022] DELETE /identity/v3/users/87689334264c4a939367f091ccfbd642 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:27.626231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.626473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.626902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.631860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2159] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:27.645954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:27.647978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:27.653968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:27.655668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:27.728465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b02d1-1caf-4712-95bd-eb5646bd57d1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:27.743325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2160] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:27.780622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:27.782179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:27.798057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:27.804515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:27.805770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.806228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.806727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.822574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:27.824483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:27.840575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2161] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:27.871383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc54a3-0779-4219-ad16-a644999b22ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:27.875658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:27.876322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:27.876617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:27.876888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:27.887634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:27.889135 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75c89d-0d00-4d26-a0c8-691fd42261c4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:27.900544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2162] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:19:28.336042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:28.338262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:28.358407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d9e74-3b62-4c81-8149-2b3b74271ba9 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:28.366073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:28.367032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.367445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.367858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] PATH_INFO: `/v3/projects/0df8bd6fe9ca418d9e295aa3f419f5de` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.369361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df8bd6fe9ca418d9e295aa3f419f5de)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:28.371570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:28.485948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df8bd6fe9ca418d9e295aa3f419f5de', 'request_id': 'req-91fb2a52-e079-47a7-80f9-530c5d3320d8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.495929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df8bd6fe9ca418d9e295aa3f419f5de', 'request_id': 'req-91fb2a52-e079-47a7-80f9-530c5d3320d8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.502990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df8bd6fe9ca418d9e295aa3f419f5de', 'request_id': 'req-91fb2a52-e079-47a7-80f9-530c5d3320d8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.518801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] The token cache is being invalidate because project 0df8bd6fe9ca418d9e295aa3f419f5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:28.519088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.519750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.522175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2163] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:28 2022] DELETE /identity/v3/projects/0df8bd6fe9ca418d9e295aa3f419f5de => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:28.529120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc524d56-4354-4900-9bb5-6b80ae3bb56c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.530002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc524d56-4354-4900-9bb5-6b80ae3bb56c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.530457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc524d56-4354-4900-9bb5-6b80ae3bb56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.563837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc524d56-4354-4900-9bb5-6b80ae3bb56c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:28.565418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc524d56-4354-4900-9bb5-6b80ae3bb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:28.634556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:28.648833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:28.650768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:28.682414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2164] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:28.690105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.690386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.690656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.729888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:28.731781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:28.742595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2165] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:28.748891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:28.765278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a6b4-fe33-4e95-b422-881306d7d7c6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:28.771900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:28.772570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.772890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.773155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.773810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:28.774103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:28.775825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:28.784007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb2a52-e079-47a7-80f9-530c5d3320d8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:28.789734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:28.791430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.791844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.792291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] PATH_INFO: `/v3/projects/5c4d567e8a2948afa9c76331c70493e9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.794204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4d567e8a2948afa9c76331c70493e9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:28.796613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:28.800566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2166] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:19:28.801619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:28.813085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e379f27-7265-4d23-88e8-a353be9163e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:28.818329 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:28.819529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.819883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.820427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.821385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:28.823583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:28.865872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4d567e8a2948afa9c76331c70493e9', 'request_id': 'req-c8a704ef-6d39-45fe-b168-10a266fb6e1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.868625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2167] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:28 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:28.872707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:28.885902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4d567e8a2948afa9c76331c70493e9', 'request_id': 'req-c8a704ef-6d39-45fe-b168-10a266fb6e1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.907474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4d567e8a2948afa9c76331c70493e9', 'request_id': 'req-c8a704ef-6d39-45fe-b168-10a266fb6e1b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.942570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] The token cache is being invalidate because project 5c4d567e8a2948afa9c76331c70493e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:19:28.943496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.944682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c8a704ef-6d39-45fe-b168-10a266fb6e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:19:28.952020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2168] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:19:28 2022] DELETE /identity/v3/projects/5c4d567e8a2948afa9c76331c70493e9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:28.955992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec59a7-0b0a-4fd1-98dd-2ed6c50f0e6a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.956411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec59a7-0b0a-4fd1-98dd-2ed6c50f0e6a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.956746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec59a7-0b0a-4fd1-98dd-2ed6c50f0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.970479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:28.985424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfd9fa-662b-45c0-9e24-4302c65c3429 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:28.985918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-daec59a7-0b0a-4fd1-98dd-2ed6c50f0e6a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:28.987995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-daec59a7-0b0a-4fd1-98dd-2ed6c50f0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:28.992353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:28.993175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:28.993716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:28.994274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:28.995365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:28.997825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:28.999532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2169] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:29.008161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.008490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.008849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.028510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2170] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:28 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:19:29.031344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:29.032430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.032840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:29.034064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:29.042036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2171] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:29.045957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.046888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eeb162-f5c0-436d-95b6-39943f886995 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.053133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.053891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.054263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.054628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.055379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.056720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c676a95-5c8b-4e56-a935-7f69868788cf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.058124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.063123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.063857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.064248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.064643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.065394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:19:29.065794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.067644 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.077055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2172] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:29.079215 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.081740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2173] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:19:29.086037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.094528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776eb85-c1e4-4a68-bad7-a75584bab8ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.097534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad04c53-1248-4dfd-862f-52c31f606986 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.100656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.101456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.101943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.102185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.102592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.102829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.103313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.103656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.103991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.104729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.105512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.106913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.117926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2174] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:29.122154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.133586 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe6243-0ad8-4ac9-8600-7737c895202b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.139535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.140233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.140539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.140813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] PATH_INFO: `/v3/projects/7a2f6acb3c3a4b24be797a677d868227/users/ff3135cc1482461caf69068b659f49e4/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.145408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2175] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:29 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:29.149276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.162818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a2f6acb3c3a4b24be797a677d868227, user_id=ff3135cc1482461caf69068b659f49e4, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.166434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094782f0-89e3-4f5a-9c5a-4afe3102848a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.187192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2176] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:29 2022] PUT /identity/v3/projects/7a2f6acb3c3a4b24be797a677d868227/users/ff3135cc1482461caf69068b659f49e4/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:29.192047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.192497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.194500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.216587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b02aaf-4ce1-43ca-9b8a-b242851f10c0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.222664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:29.224243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:29.280198 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:29.289444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.290767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.290767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.290767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.291606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.293907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.305047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2177] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:29.317350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.318998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:29.326524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2178] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:29 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:19:29.330666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.341813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd0a2-2647-42e4-ab17-3a6bdf8717c6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.347895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.348924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.348924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.349296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.349977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.351659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.362562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2179] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:29.366670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.377117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a6721-de4c-4823-83f8-fe2368670662 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.382998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.383733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.384092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.384478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.385214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.387288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.398367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a2007-6522-45b9-bc98-889bfca995d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.399577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2180] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:19:29.402753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.405994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.406831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.407108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.407374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.419558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.419983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f81aa-450c-4d5e-ae6c-e0d7ade8a968 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.421691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008a5cd-bd1b-4b25-83d4-9a2dea8cd5ea service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.427061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.428005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.428371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.429098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] PATH_INFO: `/v3/projects/821c49e74c4e4767b872c300ac829345/users/0fb97bc956b14342990b1dd5d8c7be95/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.448788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=821c49e74c4e4767b872c300ac829345, user_id=0fb97bc956b14342990b1dd5d8c7be95, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.451587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.464512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2181] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:19:29.486565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2182] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:29 2022] PUT /identity/v3/projects/821c49e74c4e4767b872c300ac829345/users/0fb97bc956b14342990b1dd5d8c7be95/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:29.497269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa724-7f3e-46e7-a6ef-6de4f25cab1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.497838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa724-7f3e-46e7-a6ef-6de4f25cab1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.498523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aa724-7f3e-46e7-a6ef-6de4f25cab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.548429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ec6aa724-7f3e-46e7-a6ef-6de4f25cab1c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:29.550002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec6aa724-7f3e-46e7-a6ef-6de4f25cab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:29.636102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2183] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:29.649657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:29.661031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:29.752948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba1c8-7fb7-4dd6-afd6-d18927bcd7cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:29.759053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:29.760194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:29.761010 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:29.761519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:29.789957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:29.792256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:29.811049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2184] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:31.059566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e59f36-15ce-4a05-bb9b-e6cec8b4f682 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:31.060101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e59f36-15ce-4a05-bb9b-e6cec8b4f682 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:31.060101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e59f36-15ce-4a05-bb9b-e6cec8b4f682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:31.091843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8e59f36-15ce-4a05-bb9b-e6cec8b4f682 None None] MFA Rules not processed for user `ff3135cc1482461caf69068b659f49e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:31.093417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8e59f36-15ce-4a05-bb9b-e6cec8b4f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:31.191580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2185] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:31.203112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:31.204937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:31.272400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65bf51-1498-47d9-bf99-f4ea501435d0 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:31.276540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:31.277156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:31.277539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:31.277854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:31.291270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:31.293272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429af90-a276-475e-98cc-252f0f1fc0c1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:31.303039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:33.835032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:33.835598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:33.835598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:33.856478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] MFA Rules not processed for user `ff3135cc1482461caf69068b659f49e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:33.857715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:33.865082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2187] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:34.490628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749371f-7153-4438-8613-f444a6c68a04 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:34.491583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749371f-7153-4438-8613-f444a6c68a04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:34.492019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749371f-7153-4438-8613-f444a6c68a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:34.520947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f749371f-7153-4438-8613-f444a6c68a04 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:34.522488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f749371f-7153-4438-8613-f444a6c68a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:34.676237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2188] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:19:34.690563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:34.692808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:34.717058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:34.853099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742ff2-0734-4750-b903-b4672fd41510 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:34.859617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:34.861277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:34.861823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:34.862438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:34.874847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:34.875387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:34.875790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:34.880181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:34.882102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:34.903271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2189] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:19:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 11:19:34.925508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] MFA Rules not processed for user `0fb97bc956b14342990b1dd5d8c7be95`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:34.927753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:35.113057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:35.182064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2190] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:35.189588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.220091 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217010a-016d-4c7d-ac60-f421edec1ac9 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.228696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.229318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.229589 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.229859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.233113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.235043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:35.326898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2191] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:19:35 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:35.331229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.355658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c011be2-fb65-4cde-8ea7-708011229bee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.506842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.508226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.508655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.509094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.509768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.512183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:35.545549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2192] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:19:35 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:19:35.550007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.564467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb76a-d59a-4427-9639-e7e6e1e4b237 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.580126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.580922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.581449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.581788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.582713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.586436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:35.603019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2193] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:35.606959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.622314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09758c2a-6eba-4c1a-bf00-c039b0e26855 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.645758 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.648845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.649619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.650285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] PATH_INFO: `/v3/projects/58c072b6f7cb4089a865ccc758faea65/users/8a45508f4ce6417abac7c23d66fbd1e0/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.728673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c072b6f7cb4089a865ccc758faea65, user_id=8a45508f4ce6417abac7c23d66fbd1e0, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.731646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:35.766822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2194] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:35 2022] PUT /identity/v3/projects/58c072b6f7cb4089a865ccc758faea65/users/8a45508f4ce6417abac7c23d66fbd1e0/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:19:35.771016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.811378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413267-c3b3-4f03-9861-ab914aad3c75 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.895992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.898754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.899239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.899711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.900604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.902831 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e8ba4-c435-46e1-8dc7-3669f9a9a5ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:35.926232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2195] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:19:35.929343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:35.947896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7647c-4f66-4ef7-bad4-dbfdd6372ef6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:35.967464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:35.968858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:35.969145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:35.970261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] PATH_INFO: `/v3/projects/58c072b6f7cb4089a865ccc758faea65/users/8a45508f4ce6417abac7c23d66fbd1e0/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:35.975643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c072b6f7cb4089a865ccc758faea65, user_id=8a45508f4ce6417abac7c23d66fbd1e0, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:35.981206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:36.014121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2196] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:19:35 2022] PUT /identity/v3/projects/58c072b6f7cb4089a865ccc758faea65/users/8a45508f4ce6417abac7c23d66fbd1e0/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:19:45.911384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:46.001160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=50,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:19:46.301787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257ab52-f306-4c47-9ac5-a918c1dd3472 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:46.302405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257ab52-f306-4c47-9ac5-a918c1dd3472 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:46.302894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257ab52-f306-4c47-9ac5-a918c1dd3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:46.357970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3257ab52-f306-4c47-9ac5-a918c1dd3472 None None] MFA Rules not processed for user `8a45508f4ce6417abac7c23d66fbd1e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:46.360081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3257ab52-f306-4c47-9ac5-a918c1dd3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:46.472102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2197] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:46.749790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:19:46.757601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:47.155337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ca20-4c2a-4ec0-9daa-f7ca6d5fac92 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:19:47.167268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:19:47.167997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:47.168612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:47.168886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:47.272199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:19:47.273836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b3405-2019-4e86-82f7-1d7849df067c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:19:47.291033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2198] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:19:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:19:50.175485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d24e5-f359-49d3-9291-5de7060fcd05 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:50.176065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d24e5-f359-49d3-9291-5de7060fcd05 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:50.176065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d24e5-f359-49d3-9291-5de7060fcd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:50.207617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fd9d24e5-f359-49d3-9291-5de7060fcd05 None None] MFA Rules not processed for user `0fb97bc956b14342990b1dd5d8c7be95`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:50.209489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd9d24e5-f359-49d3-9291-5de7060fcd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:50.223691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2199] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:19:52.570743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:19:52.571329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:19:52.571329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:19:52.598603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] MFA Rules not processed for user `f4da6dd860bb45d1a703a18c2512ffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:19:52.600720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:19:52.768752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2200] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:19:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:00.200337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=37 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:02.752848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:03.609567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665553-4534-4fd2-990e-0d06d2e5ce0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:03.610186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665553-4534-4fd2-990e-0d06d2e5ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:03.610186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665553-4534-4fd2-990e-0d06d2e5ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:03.631646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ec665553-4534-4fd2-990e-0d06d2e5ce0a None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:03.633002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec665553-4534-4fd2-990e-0d06d2e5ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:03.701844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2201] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:20:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:03.718680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:03.791305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d493741-5731-4c6b-9107-de9ae388f67a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:03.799832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:03.800513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:03.800780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:03.801183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:03.810252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:03.811878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:03.821500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2202] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:20:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:20:06.119969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806688e-45d1-40a0-a884-276c77060383 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:06.120914 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806688e-45d1-40a0-a884-276c77060383 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:06.120914 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806688e-45d1-40a0-a884-276c77060383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:06.142380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8806688e-45d1-40a0-a884-276c77060383 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:06.143687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8806688e-45d1-40a0-a884-276c77060383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:06.156976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2203] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:20:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:06.165634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:06.185502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef0bda-f1bd-47c1-a051-82e033a6045d service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:06.193154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:06.194624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:06.195329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:06.196778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:06.209233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:06.222218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0864c-5cf8-440b-b91d-695cf1acf896 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:06.248237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2204] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:20:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:20:09.290206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:09.290206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:09.291282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:09.292701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2205] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:20:09.340861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:09.342569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:09.343137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:09.370108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] MFA Rules not processed for user `9792491cfe3c489a83639e77a27bde04`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:09.371716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:09.455406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:09.459312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:09.502209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2206] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:20:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:09.539435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:09.541752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:09.562393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27435-71c9-4fa0-ab67-8892d79b4d5a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:09.581309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:09.581448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:09.581697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:09.581972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:09.595837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a298076-5721-4c0e-bc4f-687a78878fc0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:09.597245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:09.598832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:09.624999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:09.625793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:09.626073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:09.626386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] PATH_INFO: `/v3/users/b3c011df2b144f758dd286e77875bcc8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:09.627593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c011df2b144f758dd286e77875bcc8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:09.630057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2207] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:20:09.632615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:09.732784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c011df2b144f758dd286e77875bcc8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:09.745494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c011df2b144f758dd286e77875bcc8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:09.779928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c011df2b144f758dd286e77875bcc8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:09.780754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c011df2b144f758dd286e77875bcc8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:09.801770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b3dae13d-6eab-4124-bd8e-6c49c30b6e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c011df2b144f758dd286e77875bcc8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:09.809102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2208] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:20:09 2022] DELETE /identity/v3/users/b3c011df2b144f758dd286e77875bcc8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:20:10.069961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.083913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d73fb-59f0-410a-8f8f-5eab5d7822f3 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.171049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.172435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.173053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.174577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] PATH_INFO: `/v3/projects/b37fc7dacc11483ca9c6f047d3bd2553` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.178457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37fc7dacc11483ca9c6f047d3bd2553)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.183978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.186411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:10.285907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37fc7dacc11483ca9c6f047d3bd2553', 'request_id': 'req-c4b66a28-782e-4e6c-b95d-9c51023e85ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:10.292112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37fc7dacc11483ca9c6f047d3bd2553', 'request_id': 'req-c4b66a28-782e-4e6c-b95d-9c51023e85ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:10.299029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37fc7dacc11483ca9c6f047d3bd2553', 'request_id': 'req-c4b66a28-782e-4e6c-b95d-9c51023e85ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:10.318520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] The token cache is being invalidate because project b37fc7dacc11483ca9c6f047d3bd2553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:20:10.318812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:10.319526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4b66a28-782e-4e6c-b95d-9c51023e85ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:20:10.326631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2209] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:20:10 2022] DELETE /identity/v3/projects/b37fc7dacc11483ca9c6f047d3bd2553 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:20:10.328493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.328991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.329085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.355392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:10.365718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:10.380762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:10.461640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2210] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:10.469495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.470106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.470106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.492017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:10.493891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:10.506893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2211] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:20:10.511082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.522563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d683a-daad-4d5d-bb3e-ed2886bd0a02 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.543786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.543786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.543786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.543786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.544843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:20:10.545148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.547017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.562299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2212] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:20:10.566911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.577800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f9e67-a4b5-481d-a2a1-394549541cfd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.584454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.585079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.585351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.585614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.586327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.590548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.642112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2213] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:20:10 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:20:10.646392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.664399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384fb4-7760-47a6-847d-cfe62eef1ebc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.786616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.787287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.787563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.787888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.789180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.791273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.845617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2214] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:20:10 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:20:10.846566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.873417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9521-b660-4e49-a133-d70d9f76eba9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.897635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.898422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.898788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.899150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.900334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.902165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.933683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:10.934246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2215] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:20:10.949683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231c8ea-dbe5-4cde-900a-c56f06f290c3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:10.968188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:10.969168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:10.969682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:10.970154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:10.970998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:10.973386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49e861-7c73-424b-96d1-48a590f67216 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:10.992145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2216] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:20:10.993063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:11.006372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9d036-188a-49f7-abc6-7a97f855d48f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:11.017183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:11.018053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:11.018429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:11.019410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] PATH_INFO: `/v3/projects/5b45bc660add428a9d69ed918c68e58f/users/edd336f4aa5441909c67c91aaf0f1c5a/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:11.057756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b45bc660add428a9d69ed918c68e58f, user_id=edd336f4aa5441909c67c91aaf0f1c5a, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:11.061278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:11.087517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2217] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:20:10 2022] PUT /identity/v3/projects/5b45bc660add428a9d69ed918c68e58f/users/edd336f4aa5441909c67c91aaf0f1c5a/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:20:11.098673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586020a-494c-4eb1-87eb-0b7968c35b36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:11.098673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586020a-494c-4eb1-87eb-0b7968c35b36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:11.098673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586020a-494c-4eb1-87eb-0b7968c35b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:11.144837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2586020a-494c-4eb1-87eb-0b7968c35b36 None None] MFA Rules not processed for user `edd336f4aa5441909c67c91aaf0f1c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:11.150346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2586020a-494c-4eb1-87eb-0b7968c35b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:11.365844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2218] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:20:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:20:15.224929 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:15.226719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:15.285721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1667eb-8310-4757-a1d8-a7a95596ee8f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:15.289880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:15.290567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:15.291396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:15.291691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:15.301608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:15.303154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d791d07-de6f-4a95-90ed-08e3883a03f6 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:15.313386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:20:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:20:21.355047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:24.574898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:24.575479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:24.575479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:24.577146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2220] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:20:24.585554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5ecb-f7da-4fa5-b6b1-aa64716185fa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:24.586098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5ecb-f7da-4fa5-b6b1-aa64716185fa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:24.586098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5ecb-f7da-4fa5-b6b1-aa64716185fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:24.607126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-975c5ecb-f7da-4fa5-b6b1-aa64716185fa None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:24.608936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-975c5ecb-f7da-4fa5-b6b1-aa64716185fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:24.641813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:24.693768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2221] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:24.711556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:24.715624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:24.744864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae8672-ed31-4269-af1d-79a1ed35e0b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:24.778875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:24.779511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:24.779792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:24.780062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:24.791757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:24.793632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef62fd-a88a-41f8-a601-3ed5bc49974c service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:24.808821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2222] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:20:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:20:24.927905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:24.928182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:24.929408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:24.931741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2223] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:20:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:20:24.937926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2393e-cb65-4452-b674-92740938100a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:24.938403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2393e-cb65-4452-b674-92740938100a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:24.938963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2393e-cb65-4452-b674-92740938100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:24.968788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fc2393e-cb65-4452-b674-92740938100a None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:24.972735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fc2393e-cb65-4452-b674-92740938100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:25.036314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2224] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:25.051464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:25.053230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:25.077193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a240c-adf7-4d72-92ea-42471f11e2cc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:25.099407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:25.100050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:25.100369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:25.101128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:25.116391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:25.118364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cfff9-ce3b-4d9d-98e0-b065c3bdc1d7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:25.131205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2225] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:20:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:20:25.272715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd271f8-36b0-4ad0-939d-f998bbf32bdf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:25.273088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd271f8-36b0-4ad0-939d-f998bbf32bdf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:25.273260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd271f8-36b0-4ad0-939d-f998bbf32bdf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:25.274573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2226] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:20:25.279953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8fb6b-6196-4a11-bdc2-8d44c2f99c32 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:25.280236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8fb6b-6196-4a11-bdc2-8d44c2f99c32 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:25.280532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8fb6b-6196-4a11-bdc2-8d44c2f99c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:25.312065 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b3a8fb6b-6196-4a11-bdc2-8d44c2f99c32 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:25.313655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3a8fb6b-6196-4a11-bdc2-8d44c2f99c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:25.323244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2227] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:20:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:20:25.336344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd271f8-36b0-4ad0-939d-f998bbf32bdf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:25.354710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd271f8-36b0-4ad0-939d-f998bbf32bdf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:25.360132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:25.360867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:25.361207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:25.361544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:25.372698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:25.374420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:25.386365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:20:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:20:28.692563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:28.693072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:28.693072 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:28.726009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:20:28.727394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:28.786617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2229] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:20:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:20:28.808748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:20:28.810160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:20:28.896739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b8885-ea2d-448e-8508-022fbb67795d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:20:28.902139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:20:28.902751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:20:28.903027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:20:28.903290 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:20:28.912187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:20:28.913843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b6c42-79da-4607-8b32-47d32b4aa18d service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:20:28.922881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2230] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:20:38.779751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=52 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:20:38.919305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=67 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:06.494066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:06.563415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08105d-7426-40f1-810c-99b9f3cadaa6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:06.568559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:06.569263 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:06.569539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:06.569809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] PATH_INFO: `/v3/users/edd336f4aa5441909c67c91aaf0f1c5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:06.571163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd336f4aa5441909c67c91aaf0f1c5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:06.573297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:06.627611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd336f4aa5441909c67c91aaf0f1c5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:06.628176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd336f4aa5441909c67c91aaf0f1c5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:06.637513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd336f4aa5441909c67c91aaf0f1c5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:06.652091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd336f4aa5441909c67c91aaf0f1c5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:06.652912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd336f4aa5441909c67c91aaf0f1c5a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:06.668318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2231] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:21:06 2022] DELETE /identity/v3/users/edd336f4aa5441909c67c91aaf0f1c5a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:21:06.670004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:06.670854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:06.670854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:06.690052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:06.691261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:06.772393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2232] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:06.779663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:06.781305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:06.840323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4020c1-b6aa-4b37-8013-b4c283104fa3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:06.845177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:06.845792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:06.846064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:06.846328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:06.855665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:06.857124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97ac05-0811-4dc1-a135-a34a8bc1d7a1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:06.875188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2233] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:21:07.171895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.173439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:07.185682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736302-b9a9-4fa1-a51e-0dff96e45e70 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.190525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.191165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.191446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.191722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] PATH_INFO: `/v3/projects/5b45bc660add428a9d69ed918c68e58f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.192904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b45bc660add428a9d69ed918c68e58f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.194826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.249901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b45bc660add428a9d69ed918c68e58f', 'request_id': 'req-f7187eb9-0f01-4837-887f-2d7f9c9f013b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:07.255300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b45bc660add428a9d69ed918c68e58f', 'request_id': 'req-f7187eb9-0f01-4837-887f-2d7f9c9f013b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:07.264485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b45bc660add428a9d69ed918c68e58f', 'request_id': 'req-f7187eb9-0f01-4837-887f-2d7f9c9f013b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:07.276301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] The token cache is being invalidate because project 5b45bc660add428a9d69ed918c68e58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:21:07.276598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:07.277225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7187eb9-0f01-4837-887f-2d7f9c9f013b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:07.278754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2234] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:21:07 2022] DELETE /identity/v3/projects/5b45bc660add428a9d69ed918c68e58f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:21:07.285192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.285478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.285802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.306934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:07.308145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:07.326202 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:07.372119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2235] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:07.380618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.380891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.381167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.402667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:07.403884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:07.413651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2236] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:07.420576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.430910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ac942-945d-4319-89fd-c7f62bff9c40 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.436699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.437332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.437609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.437887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.438545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:21:07.438841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.440493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.452611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2237] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:21:07.457501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.467903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4ccd-5f0d-4300-b449-2877bb8fdacb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.473744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.474347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.474615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.474870 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.475498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.477396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.506006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2238] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:21:07 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:21:07.510325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.520821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c3da8-e512-4577-87fd-8add15b1df8f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.582219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.582870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.583198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.583513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.584489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.587536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.613260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2239] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:21:07 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:21:07.618225 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.631086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1f559-c551-45de-ace4-7d03e5dd56ab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.636531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.637125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.637403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.637672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.638310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.640046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.650723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2240] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:21:07.655021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.665208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc97289-c385-4fdb-bb8b-47919db2aa95 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.670261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.670915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.671131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.671390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.672025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.673650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9704f-dfa2-4de9-a8e1-f64aff618aec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.684700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2241] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:21:07.689129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.699676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec0f4c-f01b-453a-9e68-982689601239 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.706028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.706681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.706957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.707251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] PATH_INFO: `/v3/projects/9d1257b273e84ca4b116d70a79408a93/users/8337cf2411e3434f83e77599e8c48c26/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.725011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d1257b273e84ca4b116d70a79408a93, user_id=8337cf2411e3434f83e77599e8c48c26, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.727705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.743527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2242] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:21:07 2022] PUT /identity/v3/projects/9d1257b273e84ca4b116d70a79408a93/users/8337cf2411e3434f83e77599e8c48c26/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:21:07.755854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.756297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.756746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.786941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] MFA Rules not processed for user `8337cf2411e3434f83e77599e8c48c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:07.788141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:07.855894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2243] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:07.867011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:07.868850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:07.892511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:07.934199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d78f6-3c18-46c2-a5df-f0161473c7d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:07.938149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:07.938801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:07.939121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:07.939392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:07.947940 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:07.949417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:07.959079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2244] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:21:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:21:17.852775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:17.954746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=17 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:28.405385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:28.407081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:28.476853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f34e7-4f2f-4d96-b033-bfbdfef4e7e7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:28.492684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:28.492684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:28.492684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:28.492684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:28.493339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:28.565790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:28.567211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598fc04-823c-4f02-a3c2-2458897ce333 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:28.579191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2245] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:21:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:21:30.802236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:30.877442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45037b55-08e4-4dee-917a-6ea0bdd1f7c5 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:30.882339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:30.883597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:30.884541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:30.885124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] PATH_INFO: `/v3/users/8337cf2411e3434f83e77599e8c48c26` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:30.886717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8337cf2411e3434f83e77599e8c48c26)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:30.888878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:30.969682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8337cf2411e3434f83e77599e8c48c26'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:30.978121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8337cf2411e3434f83e77599e8c48c26'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:30.999022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8337cf2411e3434f83e77599e8c48c26'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:31.000098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8337cf2411e3434f83e77599e8c48c26'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:31.015119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8337cf2411e3434f83e77599e8c48c26'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:31.022525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2246] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:21:30 2022] DELETE /identity/v3/users/8337cf2411e3434f83e77599e8c48c26 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:21:31.027096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:31.027583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:31.027666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:31.047948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:31.049278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:31.111855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2247] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:21:31.122542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:31.124160 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:31.187645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360773b5-2d75-4432-93ab-e4a6d34fa3a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:31.191780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:31.192485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:31.192761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:31.193025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:31.203614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:31.208000 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:31.224223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2248] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:21:31.348408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:31.349967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:31.468471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:31.494752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224a04-60ae-4770-9279-f2f5934e5d29 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:31.502397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:31.503080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:31.503363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:31.504527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:31.508451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:31.529334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:31.531730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cc41-0d67-4617-bb0f-1a9c85c09a92 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:31.547371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2249] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:21:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:21:32.270374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:32.271921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:32.290324 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a11da-d754-4412-a831-f20d5cda5299 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:32.295088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:32.295856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:32.296008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:32.296314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] PATH_INFO: `/v3/projects/9d1257b273e84ca4b116d70a79408a93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:32.297526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d1257b273e84ca4b116d70a79408a93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:32.299497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:32.320639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:32.364610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d1257b273e84ca4b116d70a79408a93', 'request_id': 'req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:32.371783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d1257b273e84ca4b116d70a79408a93', 'request_id': 'req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:32.383939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d1257b273e84ca4b116d70a79408a93', 'request_id': 'req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:32.418046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] The token cache is being invalidate because project 9d1257b273e84ca4b116d70a79408a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:21:32.418508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:32.419925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:21:32.421939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2250] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:21:32 2022] DELETE /identity/v3/projects/9d1257b273e84ca4b116d70a79408a93 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:21:39.387181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6376-fe01-4e7d-898b-7f8252402762 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:39.387687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6376-fe01-4e7d-898b-7f8252402762 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:39.387788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f6376-fe01-4e7d-898b-7f8252402762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:39.409852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b40f6376-fe01-4e7d-898b-7f8252402762 None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:39.411157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b40f6376-fe01-4e7d-898b-7f8252402762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:39.487789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2251] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:21:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:39.546363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:21:39.548083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:39.651004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272ef28-d6ff-4f56-af7b-9ad01b5e9016 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:21:39.657292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:21:39.658482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:39.659035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:39.659740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:39.674933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:21:39.676906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a20959-e07c-442c-af63-c2e3267d0fd8 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:21:39.690247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2252] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:21:40.281605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a05c8-21ae-43d3-9181-b1cab8af6166 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:40.281936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a05c8-21ae-43d3-9181-b1cab8af6166 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:40.282240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a05c8-21ae-43d3-9181-b1cab8af6166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:40.302440 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e04a05c8-21ae-43d3-9181-b1cab8af6166 None None] MFA Rules not processed for user `3a1cd35a257f4b0f99141580c14bad48`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:40.303870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e04a05c8-21ae-43d3-9181-b1cab8af6166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:40.360773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2253] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:47.274059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111058-46f5-432d-b398-01be8b1caf1d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:47.274625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111058-46f5-432d-b398-01be8b1caf1d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:47.274841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111058-46f5-432d-b398-01be8b1caf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:47.301080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d111058-46f5-432d-b398-01be8b1caf1d None None] MFA Rules not processed for user `ff3135cc1482461caf69068b659f49e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:47.302756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d111058-46f5-432d-b398-01be8b1caf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:47.369275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2254] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:50.353674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=59 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:21:54.481059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:21:54.481059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:21:54.481059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:21:54.500550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] MFA Rules not processed for user `f4da6dd860bb45d1a703a18c2512ffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:21:54.501765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:21:54.563061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2255] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:21:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:21:57.362510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:04.556298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:43.110911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:43.111483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:43.111483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:43.131093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] MFA Rules not processed for user `2e6543b377f14949a6919af4e4e187b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:43.132299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:43.196953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2256] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:43.886951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:43.888744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:43.951376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e3600-c9e1-42f8-ba74-43f915b5e712 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:43.955298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:43.955965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:43.956283 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:43.956578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:43.958327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:44.029265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:44.030721 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:44.041400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2257] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:22:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:22:45.160855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:45.171298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a25644-b23a-4d9e-bbbc-6df8ca093445 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:45.175667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:45.176305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:45.176581 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:45.176843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:45.178764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:45.192633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:45.194066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:45.206270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2258] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:22:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:22:50.433059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:50.434904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:50.499363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb21-e565-4b25-96df-72380035f6fe service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:50.504536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:50.505182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:50.505462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:50.505729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] PATH_INFO: `/v3/users/ff3135cc1482461caf69068b659f49e4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:50.506863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3135cc1482461caf69068b659f49e4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:50.508824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:50.559042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:50.561950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3135cc1482461caf69068b659f49e4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.563019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3135cc1482461caf69068b659f49e4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.572387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3135cc1482461caf69068b659f49e4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.584441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3135cc1482461caf69068b659f49e4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.585079 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f1920836-59c7-46cb-bd96-5ff20d1f5fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3135cc1482461caf69068b659f49e4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.597074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2259] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:22:50 2022] DELETE /identity/v3/users/ff3135cc1482461caf69068b659f49e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:22:50.790266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:50.807920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ddef-93a0-4e70-b724-67f49dcda42c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:50.871453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:50.872121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:50.872449 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:50.872722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] PATH_INFO: `/v3/projects/7a2f6acb3c3a4b24be797a677d868227` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:50.873989 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a2f6acb3c3a4b24be797a677d868227)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:50.876066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:50.931519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a2f6acb3c3a4b24be797a677d868227', 'request_id': 'req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.937582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a2f6acb3c3a4b24be797a677d868227', 'request_id': 'req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.948070 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a2f6acb3c3a4b24be797a677d868227', 'request_id': 'req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.959477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] The token cache is being invalidate because project 7a2f6acb3c3a4b24be797a677d868227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:22:50.959706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.960377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-95d811ee-072d-447a-b5b1-0a4a2aa4fd0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:22:50.961887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2260] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:22:50 2022] DELETE /identity/v3/projects/7a2f6acb3c3a4b24be797a677d868227 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:22:50.968392 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:50.968694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:50.968987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:50.989453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:50.990638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:51.059988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2261] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:51.068433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.068725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.068995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.089974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:51.091182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:51.100800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2262] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:51.105346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.115986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f216b2-e75f-4414-a0be-10f828bdc2d1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.122773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.123390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.123660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.123925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.124605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:22:51.124902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.126936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.138785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2263] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:22:51.143519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.153610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bf9ec-7a8e-451f-a496-bd37a172f00c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.159273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.159877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.160182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.160480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.161110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.163064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.178777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:51.196946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2264] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:22:51 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:22:51.198970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.209103 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789cdd7-5b3d-4901-a8dc-e0bfbef0d2ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.265738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.266448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.266781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.267122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.267765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.270095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.294030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2265] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:22:51 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:22:51.298488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.310266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397ab0-1927-49bc-a3bf-213bec7536fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.316038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.316671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.316942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.317202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.317841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.319452 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.330132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2266] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:22:51.334252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.344536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a775-578b-4d58-9a9c-7e2c2c008bc8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.350284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.351081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.351372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.351633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.352299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.353878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aadf79-3238-4fe2-a98c-8ac11833b2e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.364730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2267] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:22:51.368730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.379276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f0f90-48a9-4fcf-86a8-d1a4ce3544ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.385267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.385933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.386503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.386503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] PATH_INFO: `/v3/projects/bfd671fcff7e4b24b5415a3a1b70ad0f/users/5b4c4030b5fb4381a598a54588880b75/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.404826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd671fcff7e4b24b5415a3a1b70ad0f, user_id=5b4c4030b5fb4381a598a54588880b75, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.408907 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.430049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2268] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:22:51 2022] PUT /identity/v3/projects/bfd671fcff7e4b24b5415a3a1b70ad0f/users/5b4c4030b5fb4381a598a54588880b75/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:22:51.440231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.440747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.440829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.484962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:51.487119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:51.515197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:51.565829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2269] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:51.573840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:51.575997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:51.652182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766c1a8-d08c-4afa-895d-e41e15409a14 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:51.656002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:51.656662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:51.657234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:51.657355 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:51.670022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:51.671596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d573c-079c-41af-811b-f2041ba41d7f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:51.684925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2270] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:22:53.947035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:53.948554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:54.043560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd25d5-5d82-439b-b257-f5e8e5ecc69e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:54.049820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:54.050575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:54.050971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:54.051334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:54.053184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:54.140063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:54.141741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6350ce-407f-42fa-88dd-d030b8052d96 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:54.158261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2271] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:22:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:22:57.987943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:57.990636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:57.991066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.020851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] MFA Rules not processed for user `601782d5743c47f380358021be28cc91`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:58.022080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:58.046076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2272] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:58.198494 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.199351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.199779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.233695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] MFA Rules not processed for user `5b4c4030b5fb4381a598a54588880b75`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:22:58.234922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:22:58.337056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:22:58.362342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2273] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:22:58.366921 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.378878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2288b-16ac-4241-b81c-1310d29d3007 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.386008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.386797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.387217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.387557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.388382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.390551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.438241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2274] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:22:58 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:22:58.438865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.451280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02fb6b-82a5-45b4-b2b5-11b9107c2e73 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.545064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.545684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.545960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.546230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.546867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.549057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.575337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2275] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:22:58 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:22:58.576555 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.587830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ef2de-dc4d-415e-94bd-82b0b20ff701 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.592713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.593325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.593603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.593876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.594622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.596321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.607072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2276] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:22:58.611371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.621335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c34d5-2498-47b8-826b-711473c153e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.626146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.626801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.627078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.627345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] PATH_INFO: `/v3/projects/59b2dd90807744e0a327315f3725605e/users/f404b89b457643c381d6afaed16e8c3c/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.646451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b2dd90807744e0a327315f3725605e, user_id=f404b89b457643c381d6afaed16e8c3c, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.649432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.671435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2277] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:22:58 2022] PUT /identity/v3/projects/59b2dd90807744e0a327315f3725605e/users/f404b89b457643c381d6afaed16e8c3c/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:22:58.675478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.686609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f52a86-81af-425e-a2c2-27f92d504c29 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.778794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.779451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.779841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.780069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.780889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.782622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.799004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:22:58.799291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2278] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:22:58.811148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba017-68d9-4d83-b077-1684ec8151d7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:22:58.816623 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:22:58.822936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:22:58.823303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:22:58.823587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] PATH_INFO: `/v3/projects/59b2dd90807744e0a327315f3725605e/users/f404b89b457643c381d6afaed16e8c3c/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:22:58.825713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b2dd90807744e0a327315f3725605e, user_id=f404b89b457643c381d6afaed16e8c3c, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:22:58.828727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:22:58.842724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2279] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:22:58 2022] PUT /identity/v3/projects/59b2dd90807744e0a327315f3725605e/users/f404b89b457643c381d6afaed16e8c3c/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:23:04.145381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:04.147941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:04.179925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:04.240453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309281a0-59fe-4db2-9236-7e517da91634 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:04.349302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:04.351316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:04.351784 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:04.352735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:04.355543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:04.459941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:04.461529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a71e54-a2aa-467b-92e0-858417404eac service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:04.478773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2280] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:04.480321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:04.482601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:04.508314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de6500-bb82-481f-899d-9b3bc5af185a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:04.520478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:04.521339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:04.521762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:04.522174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:04.542303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:04.544243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:04.557402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2281] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:05.353261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:05.353857 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:05.353955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:05.387315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] MFA Rules not processed for user `f404b89b457643c381d6afaed16e8c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:23:05.388580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:05.505740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2282] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:23:05.518630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:05.614487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d12496-b5ce-4e7c-9a6f-3a341020b02f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:05.622349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:05.623229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:05.623649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:05.624085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:05.664297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:05.711172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:05.713108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3985d-d808-494d-82cd-7bd805949ef0 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:05.724185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2283] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:13.602619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:13.606600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:13.648125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:13.707476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee18ead-e710-4282-a024-0c786a21ef9a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:13.715170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:13.715170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:13.715170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:13.715170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:13.716835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:13.727196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:13.728669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:13.759171 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2284] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:15.719293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=14 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:20.941537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:20.942164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:20.942164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:20.963132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] MFA Rules not processed for user `f4da6dd860bb45d1a703a18c2512ffe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:23:20.964381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:20.971918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2285] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:23:21.313610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:21.322971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809b56-78ee-43cd-8801-6a8b48215906 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:21.327067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:21.327653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:21.327918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:21.328177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:21.330039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:21.340536 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:21.341956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf75ad9-a0d7-4293-b4f4-b3235f9f6552 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:21.350245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2286] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:21.748600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:21.750339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:21.827722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec5316-9c4a-4250-b312-655a54dfd8f5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:21.832819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:21.833446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:21.833718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:21.833987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:21.836543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:21.849278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:21.851139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:21.861777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2287] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:22.362547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:22.363152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:22.363152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:22.382074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] MFA Rules not processed for user `5b4c4030b5fb4381a598a54588880b75`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:23:22.383500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:22.391076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2288] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:23:22.761865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:22.773083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa206d-80c9-4443-859f-733e87e9a91b service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:22.778861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:22.779546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:22.779804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:22.780217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:22.783097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:22.794268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:22.795906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:22.805401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2289] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:24.814237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:24.832007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d9324-b0f6-4437-a2d4-1312479bd854 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:24.838932 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:24.839978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:24.840435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:24.841060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:24.843220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:24.857043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:24.858838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:24.880368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2290] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:26.223910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:26.238261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff77d-0268-4290-9b5b-0ea80126beb1 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:26.244463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:26.245154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:26.245433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:26.245703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:26.247549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:26.261606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:26.264381 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dae49d-7088-485f-a33c-4c70269f4cb4 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:26.276543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:29.875646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:29.877149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:29.891214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f1cea-3e6b-447e-b5ae-236cd0651a53 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:29.897677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:29.898513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:29.898787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:29.899075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:29.900836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:29.914656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:29.916304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:29.930418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2292] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:30.096630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:30.097129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:30.097246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:30.116535 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] MFA Rules not processed for user `3a1cd35a257f4b0f99141580c14bad48`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:23:30.117783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:30.177339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2293] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:23:39.923537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:40.168826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=72 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:40.594115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:40.606383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bc2d9-54c2-499d-85b9-646e54d4b149 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:40.613338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:40.613798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:40.614072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:40.614331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:40.616483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:23:40.677996 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:40.679617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd687f3-92f3-4aa1-a974-f7a4e351e77e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:40.689374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2294] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:43.052735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:23:43.073483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d0d0-be44-4fe5-a822-7c2ada9c1279 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:23:43.077174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:23:43.077786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:23:43.078058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:23:43.078322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:23:43.087700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:23:43.089297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:23:43.098829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2295] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:23:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:23:50.685270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:23:53.094851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:02.796944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f60c1-6c00-47f8-be68-05a98655baac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:02.797446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f60c1-6c00-47f8-be68-05a98655baac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:02.797446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f60c1-6c00-47f8-be68-05a98655baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:02.818054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2f6f60c1-6c00-47f8-be68-05a98655baac None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:02.819487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f6f60c1-6c00-47f8-be68-05a98655baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:02.884686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2296] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:02.933793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:02.936183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:03.011581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba94c8-dd81-42a5-9d7e-e7b4e633a487 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:03.017674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:03.018278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:03.018555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:03.018824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:03.027342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:03.028977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd6663-9040-42db-a016-dffaab76c580 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:03.037803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2297] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:24:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:24:12.872924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:13.034500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:16.716956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:16.717570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:16.717570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:16.718915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2298] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:24:16.724813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19f37-5877-46eb-b917-cd090dbc4f67 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:16.725272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19f37-5877-46eb-b917-cd090dbc4f67 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:16.725372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f19f37-5877-46eb-b917-cd090dbc4f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:16.746289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3f19f37-5877-46eb-b917-cd090dbc4f67 None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:16.747493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3f19f37-5877-46eb-b917-cd090dbc4f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:16.757976 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2299] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:16.768966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:16.771293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:16.838066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dc42-3fcd-4cf8-83e9-6fd4016fbc96 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:16.843636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:16.844524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:16.844861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:16.845360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:16.866181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:16.866181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c9f8b-f562-4fbc-ac72-79a5f57f1ea7 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:16.871118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:24:16.943532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475927b-7c36-4aea-b4dd-0bd88081af90 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:16.943741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475927b-7c36-4aea-b4dd-0bd88081af90 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:16.943974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475927b-7c36-4aea-b4dd-0bd88081af90 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:16.946130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2301] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:24:16.952481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f883-ee6f-4915-aa33-d865dca6ab2b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:16.953867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f883-ee6f-4915-aa33-d865dca6ab2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:16.954142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f883-ee6f-4915-aa33-d865dca6ab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:16.976681 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8646f883-ee6f-4915-aa33-d865dca6ab2b None None] MFA Rules not processed for user `f3216e5d1c354d8e8d5c1fbf449f882c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:16.977899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8646f883-ee6f-4915-aa33-d865dca6ab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:16.985570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2302] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:24:16.997557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475927b-7c36-4aea-b4dd-0bd88081af90 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:17.010100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475927b-7c36-4aea-b4dd-0bd88081af90 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:17.014427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:17.015029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:17.015604 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:17.016047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:17.023646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:17.025171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90112956-b1c0-457a-a1de-3707b44dba35 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:17.033676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:17.137039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4fb-ad1f-4811-a292-406063e1d3a4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:17.137397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4fb-ad1f-4811-a292-406063e1d3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:17.138238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4fb-ad1f-4811-a292-406063e1d3a4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:17.140236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2304] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 9 11:24:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 11:24:17.146003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2115c-f6b2-40a7-85ae-ce25e96a496d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:17.146280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2115c-f6b2-40a7-85ae-ce25e96a496d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:17.146578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2115c-f6b2-40a7-85ae-ce25e96a496d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:17.167775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-eff2115c-f6b2-40a7-85ae-ce25e96a496d None None] MFA Rules not processed for user `fbbd789ff1f244eb98d2e5dd001a81c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:17.169032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eff2115c-f6b2-40a7-85ae-ce25e96a496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:17.180127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2305] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 9 11:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:17.191270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72e4fb-ad1f-4811-a292-406063e1d3a4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:17.203730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72e4fb-ad1f-4811-a292-406063e1d3a4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:17.207476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:17.208066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:17.208361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:17.208625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:17.219341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:17.220834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:17.231325 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2306] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:19.666160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:19.666648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:19.666648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:19.687594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:19.688830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:19.697674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2307] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:24:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:19.712815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:19.725045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae933-cff4-474b-bdb2-844ac5123653 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:19.730166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:19.730761 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:19.731043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:19.731306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:19.739911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:19.741596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32960579-a25c-4861-8a85-fdff4733e125 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:19.750057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2308] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:24:25.703783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:25.713524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3eb539-7ed8-4b2a-b1ef-55f8897036af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:25.717095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:25.717778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:25.718047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:25.718406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:25.720466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:25.731291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:25.732740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:25.742491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2309] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:29.746546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:35.738445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=21 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:40.031212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:40.031824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:40.032995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:40.062314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:40.063605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:40.070796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2310] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:40.086627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:40.100795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47079-2c5c-4493-bed9-824c4b372cab service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:40.106325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:40.107606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:40.107890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:40.108661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:40.119104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:40.120713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:40.131983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2311] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:24:45.676508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:45.686132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932827fe-5497-4eb7-968a-a4978d6d7d1d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:45.689895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:45.690648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:45.690966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:45.691417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:45.693421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:45.763534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:45.765037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:45.775750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2312] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:49.541300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:49.542339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:49.561119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999168-2a6d-4155-b6dd-88f7cc187200 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:49.568487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:49.569874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:49.570442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:49.570920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:49.573748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:49.590892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:49.593037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:49.609124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2313] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:49.615732 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:49.617386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:49.634427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a9369-b288-45d8-be18-41279a451dcd service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:49.639523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:49.640162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:49.640468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:49.640739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] PATH_INFO: `/v3/users/601782d5743c47f380358021be28cc91` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:49.641890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601782d5743c47f380358021be28cc91)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:49.643861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:49.707966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601782d5743c47f380358021be28cc91'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:49.714892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601782d5743c47f380358021be28cc91'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:49.726393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601782d5743c47f380358021be28cc91'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:49.727127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601782d5743c47f380358021be28cc91'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:49.739268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f0ad284-0612-4c85-a506-803e2ba1f502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601782d5743c47f380358021be28cc91'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:49.745104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2314] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:49 2022] DELETE /identity/v3/users/601782d5743c47f380358021be28cc91 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:49.943835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:49.954850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615c27c-2816-4d34-9c19-31c9481384f9 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.026459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.027102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.027379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.027647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] PATH_INFO: `/v3/projects/06dd1efa43b043cf84f9808305d97bf0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.028820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=06dd1efa43b043cf84f9808305d97bf0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.030716 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.078139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06dd1efa43b043cf84f9808305d97bf0', 'request_id': 'req-57e88873-7624-4406-a254-5b97c29bacbc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:50.083533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06dd1efa43b043cf84f9808305d97bf0', 'request_id': 'req-57e88873-7624-4406-a254-5b97c29bacbc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:50.089597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06dd1efa43b043cf84f9808305d97bf0', 'request_id': 'req-57e88873-7624-4406-a254-5b97c29bacbc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:50.104433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] The token cache is being invalidate because project 06dd1efa43b043cf84f9808305d97bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:50.104739 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:50.105390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-57e88873-7624-4406-a254-5b97c29bacbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:50.107089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2315] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:49 2022] DELETE /identity/v3/projects/06dd1efa43b043cf84f9808305d97bf0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:50.112411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.112411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.112987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.138785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:50.141150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:50.201805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:50.219765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2316] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:50.223761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.224050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.224320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.244575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:50.245766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:50.254798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2317] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:50.259371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.269803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa0bd0-e0e4-44c9-8a7d-0d6d6abd4cba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.275093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.275671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.275975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.276241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.276931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:24:50.277232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.278923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.291987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2318] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:24:50.296092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.307094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1c73f-9b05-441c-a69d-6b4a09a201f8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.313603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.314512 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.315226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.315578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.316240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.318178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.351821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2319] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:50.356017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.369251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab6f10-749f-4b6c-b5ab-5a6eb62c474d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.434399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.435210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.435646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.436066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.436906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.439005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.465519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2320] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:50.469948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.483747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312bf5d-331d-410f-ac30-8182c430a07d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.489917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.490538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.490807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.491184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.491845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.493604 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.506254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2321] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:50.514520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.528476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01940e-4278-492c-a54a-cda17c0fe5f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.534155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.534756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.535386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.535822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.536654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.538254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.557874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2322] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:50.558749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.560498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:50.561238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.572060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f4316-bbbe-49c2-bb1c-7a933d2904c1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.577072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.577730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.578002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.578265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] PATH_INFO: `/v3/projects/7d8940539a4c475eb3c430e8777c1b7d/users/a2f0991165394a06860e8811152c0699/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.594017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8940539a4c475eb3c430e8777c1b7d, user_id=a2f0991165394a06860e8811152c0699, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.595236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:50.597609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7574833-2cb1-4ff3-9839-dfe4acc0b785 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.618336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2323] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:50 2022] PUT /identity/v3/projects/7d8940539a4c475eb3c430e8777c1b7d/users/a2f0991165394a06860e8811152c0699/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:50.630427 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.630712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.630974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.631765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c26ee-b373-40e6-8b1a-568606180ea7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.662048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] MFA Rules not processed for user `a2f0991165394a06860e8811152c0699`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:50.663520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:50.685943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.686972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.687249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.687509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.690047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:50.733886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2324] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:24:50.738266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.784852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.787080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.797032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:50.824224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2325] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:50.846082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d61027-edeb-43ee-a6d2-8b004fc556f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.852578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.854282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.854559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.854843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.855800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:50.859270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:50.902105 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2326] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:50.906593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:50.929804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203a95b-4ede-41cb-ba1d-e27c3b6d8637 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:50.996693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:50.997464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:50.997801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:50.998136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:50.998850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.000940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.024801 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2327] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:50 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:51.029055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.039446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4add1-13e3-45af-ab3c-02d9a409e374 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.043930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.044572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.044846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.045106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.045850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.047448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.057924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2328] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:51.062732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.075344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9e92f-2f14-4a22-98f4-c8dd13445e0d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.082110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.083063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.083587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.084186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] PATH_INFO: `/v3/projects/51c98c5986574883b404b95d3c4ea665/users/7aacb3a82d184cff8833397fca84075e/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.101511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c98c5986574883b404b95d3c4ea665, user_id=7aacb3a82d184cff8833397fca84075e, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.104420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.108050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:51.122108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2329] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:51 2022] PUT /identity/v3/projects/51c98c5986574883b404b95d3c4ea665/users/7aacb3a82d184cff8833397fca84075e/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:51.125980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.136322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c2b4d-2a55-439e-a8c7-4b3b856897ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.206407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.207343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.207775 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.208231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.209485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.211617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e021-fb09-434a-b867-083b25aaaf78 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.226101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2330] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:51.230021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.242344 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbd7da-dc9d-40db-b68f-b4060166274d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.248500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.249276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.249622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.249964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] PATH_INFO: `/v3/projects/51c98c5986574883b404b95d3c4ea665/users/7aacb3a82d184cff8833397fca84075e/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.252520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c98c5986574883b404b95d3c4ea665, user_id=7aacb3a82d184cff8833397fca84075e, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.255709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.269927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2331] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:51 2022] PUT /identity/v3/projects/51c98c5986574883b404b95d3c4ea665/users/7aacb3a82d184cff8833397fca84075e/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:51.280702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.281359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.281654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.321630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] MFA Rules not processed for user `7aacb3a82d184cff8833397fca84075e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:51.323876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:51.411385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2332] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:51.423146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.425255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:51.486251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20947f5d-794e-4479-aab8-99f84b159e7d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.490448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.491132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.491468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.491797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.501223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.502663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2531-0d9e-453b-8247-87f7835594c2 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.514288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:51.582757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.612889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:51.648314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899203eb-587c-4bae-bf9f-faa1d4537fa6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.652873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.653528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.653798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.654028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] PATH_INFO: `/v3/users/a2f0991165394a06860e8811152c0699` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.655181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f0991165394a06860e8811152c0699)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.657141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.711468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f0991165394a06860e8811152c0699'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:51.717961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f0991165394a06860e8811152c0699'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:51.726348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f0991165394a06860e8811152c0699'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:51.727435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f0991165394a06860e8811152c0699'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:51.741508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f0991165394a06860e8811152c0699'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:51.748308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2334] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:51 2022] DELETE /identity/v3/users/a2f0991165394a06860e8811152c0699 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:24:51.753078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.753587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.753710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.779606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:51.780847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:51.848149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2335] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:51.859108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:51.860558 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:51.933278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d2ef-cbd0-4357-967f-f7fe0f750851 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:51.937699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:51.938351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:51.938649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:51.938911 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:51.952444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:51.955300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:51.966774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2336] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:52.015802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:52.026358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d3637-e20c-4fa5-9e17-8e7cdb2b4a6e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:52.031092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:52.031746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:52.031981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:52.032171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:52.034708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:52.049358 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:52.051122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:52.064404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2337] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:52.552687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:52.554697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:52.569813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cd7ca-cf3b-4671-b79e-2451a5a3c59a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:52.577657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:52.579075 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:52.579403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:52.579680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] PATH_INFO: `/v3/projects/7d8940539a4c475eb3c430e8777c1b7d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:52.581815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8940539a4c475eb3c430e8777c1b7d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:52.584326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:52.651200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:52.656054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8940539a4c475eb3c430e8777c1b7d', 'request_id': 'req-b63f4b95-3c59-4147-9948-30642fc1e532'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.663432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8940539a4c475eb3c430e8777c1b7d', 'request_id': 'req-b63f4b95-3c59-4147-9948-30642fc1e532'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.677318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8940539a4c475eb3c430e8777c1b7d', 'request_id': 'req-b63f4b95-3c59-4147-9948-30642fc1e532'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.705037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] The token cache is being invalidate because project 7d8940539a4c475eb3c430e8777c1b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:52.705315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.708668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.710931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2338] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:52 2022] DELETE /identity/v3/projects/7d8940539a4c475eb3c430e8777c1b7d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:52.715058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:52.725111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ae468-dd93-44c8-ae5d-a64ec7e1cd20 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:52.798924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:52.799766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:52.800238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:52.800817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] PATH_INFO: `/v3/users/7aacb3a82d184cff8833397fca84075e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:52.802205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aacb3a82d184cff8833397fca84075e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:52.804612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:52.820684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:52.867053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aacb3a82d184cff8833397fca84075e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.867772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aacb3a82d184cff8833397fca84075e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.878898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aacb3a82d184cff8833397fca84075e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.898514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aacb3a82d184cff8833397fca84075e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.899382 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-601f9849-60b3-4361-990d-ceb6d8171a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aacb3a82d184cff8833397fca84075e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:52.929614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2339] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:52 2022] DELETE /identity/v3/users/7aacb3a82d184cff8833397fca84075e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:53.765762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:53.767783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:53.787395 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4b95-3c59-4147-9948-30642fc1e532 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:53.887922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:53.890565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:53.891050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:53.891483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] PATH_INFO: `/v3/projects/51c98c5986574883b404b95d3c4ea665` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:53.893158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c98c5986574883b404b95d3c4ea665)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:53.895861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:53.989385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c98c5986574883b404b95d3c4ea665', 'request_id': 'req-d710bea0-f088-4c7e-9cfc-ce4b492437bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.001407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c98c5986574883b404b95d3c4ea665', 'request_id': 'req-d710bea0-f088-4c7e-9cfc-ce4b492437bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.021236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c98c5986574883b404b95d3c4ea665', 'request_id': 'req-d710bea0-f088-4c7e-9cfc-ce4b492437bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.042773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] The token cache is being invalidate because project 51c98c5986574883b404b95d3c4ea665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:54.043062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.043877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d710bea0-f088-4c7e-9cfc-ce4b492437bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.045793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2340] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:53 2022] DELETE /identity/v3/projects/51c98c5986574883b404b95d3c4ea665 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:54.050500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.051058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.052241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.084975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:54.089589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:54.196354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2341] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:54.204825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.205143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.205518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.243286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:54.244728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:54.262357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2342] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:54.268121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.280172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0497ebf-2938-4daf-b40e-209021207f35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.284979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.285625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.285903 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.286169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.286821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:24:54.287121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.288819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.306737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2343] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:24:54.307693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.326459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9c37f-5802-4e6a-8a34-3ae7d8c35a92 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.337409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.338499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.338499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.338682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.339203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.341164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.405055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2344] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:54.411345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.424434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b17355-ba42-4d68-be4f-c988644c8f18 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.474722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.476578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:54.542378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.543006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.543800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.544097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.544788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.548905 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.552478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:54.595856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2345] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/users => generated 305 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:54.597563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.609534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028eaa8-1cb4-4564-96d2-622774db4ac0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.615592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.616209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.616518 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.616795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.617497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.618045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d7348-065f-46b7-a947-5e68152f414b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.619157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.622643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:54.624673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.625583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.626083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.626394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] PATH_INFO: `/v3/users/f4da6dd860bb45d1a703a18c2512ffe4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.627935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4da6dd860bb45d1a703a18c2512ffe4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.631379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.635248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2346] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:24:54.636410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.656487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751cbc-e94e-4998-a5df-874192aa40fa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.664562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.665244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.665683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.665961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.667446 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.669214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.682886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2347] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:54.688158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.702645 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c510b6-28d1-4bfd-9b3a-2d7e4e701e2c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:54.711456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:54.712128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.712527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.712797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] PATH_INFO: `/v3/projects/c77d348619b2492fb4049fdd6909f416/users/d4804971688d4597be96394e7ffc173e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.716109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4da6dd860bb45d1a703a18c2512ffe4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.730128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77d348619b2492fb4049fdd6909f416, user_id=d4804971688d4597be96394e7ffc173e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:54.732824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bfc31-ae95-4aba-810c-81432594ab43 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:54.734113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4da6dd860bb45d1a703a18c2512ffe4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.745447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4da6dd860bb45d1a703a18c2512ffe4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.746571 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4da6dd860bb45d1a703a18c2512ffe4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.756352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2348] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:54 2022] PUT /identity/v3/projects/c77d348619b2492fb4049fdd6909f416/users/d4804971688d4597be96394e7ffc173e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:54.766565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:54.766846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:54.767121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:54.774494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4da6dd860bb45d1a703a18c2512ffe4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:54.782898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2349] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:54 2022] DELETE /identity/v3/users/f4da6dd860bb45d1a703a18c2512ffe4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:24:54.806963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] MFA Rules not processed for user `d4804971688d4597be96394e7ffc173e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:54.808380 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:54.891089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2350] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:54.895779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:54.898561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:54.997905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9661cb-1fa7-4392-95b2-88a16963651f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.006910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.008256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.008431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.008601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.010925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.011999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.014250 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.036653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.047114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba14b1-1a8f-468e-86b6-b4a0dfce0777 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.055279 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.057121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.057618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.058115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] PATH_INFO: `/v3/projects/cba1dd0b689b4fff847b6fd3787b143c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.059747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba1dd0b689b4fff847b6fd3787b143c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.061996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.089239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2351] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:54 2022] POST /identity/v3/projects => generated 407 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:55.095043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.173310 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba1dd0b689b4fff847b6fd3787b143c', 'request_id': 'req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:55.194697 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba1dd0b689b4fff847b6fd3787b143c', 'request_id': 'req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:55.198432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:55.210678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba1dd0b689b4fff847b6fd3787b143c', 'request_id': 'req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:55.240823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] The token cache is being invalidate because project cba1dd0b689b4fff847b6fd3787b143c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:55.241205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:55.243346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9aa9bd97-4c69-43eb-8275-7aa3bc481a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:55.246350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2352] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:55 2022] DELETE /identity/v3/projects/cba1dd0b689b4fff847b6fd3787b143c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:24:55.252408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21efb-b799-4f69-befd-7c1d8f811cc0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.252728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21efb-b799-4f69-befd-7c1d8f811cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.253950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21efb-b799-4f69-befd-7c1d8f811cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.257774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53261e-f3cd-475b-9621-5a8e3f337966 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.297978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b3b21efb-b799-4f69-befd-7c1d8f811cc0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:55.301899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3b21efb-b799-4f69-befd-7c1d8f811cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.339406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:55.352488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.353424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.355104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.355104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.356728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.361763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.383703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2353] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:55.393826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2354] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/users => generated 310 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:55.398273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.399736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.400012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.400296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.402001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.415176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6c5a2-4910-4a38-9771-b05ca4177806 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.421752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.422116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:55.422661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.423241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.423419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.423772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.424610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.426637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.433056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2355] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:24:55.437145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.446742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2356] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:55.451341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.453587 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c074f-45df-46d7-ad3f-18cd7d0ac29d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.462378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.463097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.463449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.463780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.464081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732b1f6-065a-4330-8f3b-bed9cfeb7817 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.464640 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:24:55.465013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.466816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.469588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.471102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.471595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.472025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] PATH_INFO: `/v3/projects/e7c66ab9925b4542be68b3d66ed68687/users/291d1975c452459eb1b485e045e52e80/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.481784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2357] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:24:55.487457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.495258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c66ab9925b4542be68b3d66ed68687, user_id=291d1975c452459eb1b485e045e52e80, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.498240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.498663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af99d2-5d51-41a3-92e7-25ac7ef734fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.504971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.505659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.505912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.506338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.506899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.509161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.517359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2358] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:55 2022] PUT /identity/v3/projects/e7c66ab9925b4542be68b3d66ed68687/users/291d1975c452459eb1b485e045e52e80/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:55.521623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.542756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2359] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:55.546736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.614139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9644d63-334c-4e61-a5aa-34acba81b5af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.619477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb62a5-4491-4243-a21b-a9eb30779a22 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.625467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.626105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.626387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.626665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.627301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.629401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.645137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.646107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.646383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.646648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.647308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.650428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.665432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2360] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:55.666431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.667335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2361] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:55.668177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.679337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a257a-7ba5-48a6-80b6-a7ed1b6101e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.682286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94179bfb-de97-4389-9dd1-f1ec123927a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.684005 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.684800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.685216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.685475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.686324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.688034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.690193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.691542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.692208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.693090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] PATH_INFO: `/v3/projects/e7c66ab9925b4542be68b3d66ed68687/users/291d1975c452459eb1b485e045e52e80/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.696648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c66ab9925b4542be68b3d66ed68687, user_id=291d1975c452459eb1b485e045e52e80, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.701172 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3d0a7-b5d7-4b82-9829-aeb34a70bda7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.702947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2362] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:55.704132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.715618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53494d95-2fec-4181-97a1-d8fa42050a5e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.716899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2363] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:55 2022] PUT /identity/v3/projects/e7c66ab9925b4542be68b3d66ed68687/users/291d1975c452459eb1b485e045e52e80/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:55.731756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.731756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.731756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.773733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] MFA Rules not processed for user `291d1975c452459eb1b485e045e52e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:55.777136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.794454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:55.818211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.818838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.819117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.819389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.820046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.821704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.838967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2364] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:55.839485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.850616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c828-485a-409a-8084-a46e3753a462 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:55.855972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:55.856679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.856956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.857227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] PATH_INFO: `/v3/projects/68f784e682994643ab5d25c50efd60a4/users/97a90663fe3d49a2a426efe654a546b8/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.860681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:55.873476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2365] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:24:55.878188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f784e682994643ab5d25c50efd60a4, user_id=97a90663fe3d49a2a426efe654a546b8, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:55.880951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6808b-fced-41ba-8583-468c9677f8a9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:55.886673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:55.888959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.897860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2366] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:55 2022] PUT /identity/v3/projects/68f784e682994643ab5d25c50efd60a4/users/97a90663fe3d49a2a426efe654a546b8/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:55.913172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:55.913816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:55.914584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:55.950728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] MFA Rules not processed for user `97a90663fe3d49a2a426efe654a546b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:55.951823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:55.994839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4877e-76b4-4331-bfd2-2ca090f9a8ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.039502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2367] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:56.043894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.077069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.077842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.078124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.078388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.135489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f7f27-39e5-42e3-b741-b19b6abdfd68 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.159099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.159792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.160051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.160351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.161063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.163249 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.186025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.190253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.219314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2369] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:56 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:24:56.222630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2369] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:55 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:24:56.224825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.341114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d12905-5b0a-45ff-b2bb-91c3eaea9553 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.351610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.352527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.352909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.353280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.354024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.357275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.371038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:56.402610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2370] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:56 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:56.410341 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.427041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fadf7-4fc5-471b-b834-7198e5f78fba service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.433461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.434269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.434623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.434963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.435686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.437507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.455063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2371] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:56.456291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.486706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05390-4cba-4526-8650-507dec127f90 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.492869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.493558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.495671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.500431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:56.500713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.500922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] PATH_INFO: `/v3/projects/21e73f4acef24c9db1fcd57187107fb8/users/ce2ff606bb3845ffbd3a645427cd947b/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.527802 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e73f4acef24c9db1fcd57187107fb8, user_id=ce2ff606bb3845ffbd3a645427cd947b, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.535393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.562282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2372] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:56 2022] PUT /identity/v3/projects/21e73f4acef24c9db1fcd57187107fb8/users/ce2ff606bb3845ffbd3a645427cd947b/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:56.572873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.603763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110beebe-b6d2-436d-8e86-19b217df8c8d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.693941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.694567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.694847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.695115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.697093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:56.716662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d852cef-9a7f-4ae8-ba63-ca7b13eb6ca6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.722129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.722756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.735757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.736073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.736858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.739097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.759267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2373] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:56.763583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.767810 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.770090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.787854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a58034-a6d8-4928-96b6-a5ed27dd6158 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.798132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2374] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:24:56.799142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.800723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:56.802522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.803374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.803805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.804229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] PATH_INFO: `/v3/projects/21e73f4acef24c9db1fcd57187107fb8/users/ce2ff606bb3845ffbd3a645427cd947b/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.807036 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e73f4acef24c9db1fcd57187107fb8, user_id=ce2ff606bb3845ffbd3a645427cd947b, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.813181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428ec0a-c876-4445-aebf-d1a569eddb24 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.816245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb362ced-da46-4c2a-8687-fd540260af2b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:56.822827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:56.823539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.823816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.824085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] PATH_INFO: `/v3/users/3a1cd35a257f4b0f99141580c14bad48` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.825266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1cd35a257f4b0f99141580c14bad48)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:56.832297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2375] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:56 2022] PUT /identity/v3/projects/21e73f4acef24c9db1fcd57187107fb8/users/ce2ff606bb3845ffbd3a645427cd947b/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:56.832481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:56.839084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:56.839364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:56.839692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:56.874439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] MFA Rules not processed for user `ce2ff606bb3845ffbd3a645427cd947b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:56.875714 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:56.897095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1cd35a257f4b0f99141580c14bad48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:56.904879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1cd35a257f4b0f99141580c14bad48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:56.917224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1cd35a257f4b0f99141580c14bad48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:56.918170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1cd35a257f4b0f99141580c14bad48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:56.929040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:56.938879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1cd35a257f4b0f99141580c14bad48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:56.949018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2376] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:56 2022] DELETE /identity/v3/users/3a1cd35a257f4b0f99141580c14bad48 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:56.976131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2377] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:24:56.987349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:56.989146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:57.054107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8230f14-d072-4543-91ce-6be7d8d727d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.058482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.059190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.059587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.060060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.160015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.162080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:57.181512 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.183339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3667b-863d-4259-9e28-faf6b19f8df1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.203132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2378] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:57.219060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:57.253404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c800ae7-cc3d-4532-96ab-c5b3e43bae80 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.313477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.314105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.314376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.314675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] PATH_INFO: `/v3/projects/45bb3a603ddc4d3b8042b6ae687c896e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.316018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45bb3a603ddc4d3b8042b6ae687c896e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.318314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.366211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45bb3a603ddc4d3b8042b6ae687c896e', 'request_id': 'req-6f89cf42-dd68-4429-bdb8-0300bed518c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:57.371946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45bb3a603ddc4d3b8042b6ae687c896e', 'request_id': 'req-6f89cf42-dd68-4429-bdb8-0300bed518c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:57.378388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45bb3a603ddc4d3b8042b6ae687c896e', 'request_id': 'req-6f89cf42-dd68-4429-bdb8-0300bed518c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:57.394533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] The token cache is being invalidate because project 45bb3a603ddc4d3b8042b6ae687c896e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:57.394821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:57.395453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f89cf42-dd68-4429-bdb8-0300bed518c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:57.397933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2379] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:57 2022] DELETE /identity/v3/projects/45bb3a603ddc4d3b8042b6ae687c896e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:24:57.402314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.402597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.402896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.426850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:57.428071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:57.496560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2380] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:57.505292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.505579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.505853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.527247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:57.528502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:57.537192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2381] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:57.541583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.552500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4674bf3-30a3-4fce-a640-23521671a1cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.558268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.559028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.559309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.559597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.560249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:24:57.560587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.562211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.574719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2382] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:24:57.579075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.588926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd25627-4022-46d8-8e88-1ffd9eeb9ccd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.593798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.594410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.594681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.594948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.595573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.597869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.637064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2383] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:57.638150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.649345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe9c1a-ff93-4e7c-acdc-85d6e6db00da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.669849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:57.723401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.724020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.724388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.724694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.725280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.727191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.751851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2384] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:24:57 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:24:57.756131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.767333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadd1a9-7551-4ae6-aa62-2ec29c59a0bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.772882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.773493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.773768 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.774080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.774725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.776334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.788487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2385] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:57.792633 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.803045 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ca9f-51d9-4b63-b8b2-32e5c20d5d3d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.807715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.808361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.808637 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.808898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.809529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.811115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3af98-32e7-4288-a798-9b939e19b484 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.822293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2386] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:24:57.826264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:57.837453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2000-4f81-450c-8cbd-66667046c28b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:57.842810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:57.843469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.843745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.844006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] PATH_INFO: `/v3/projects/01654f0190a14e7691d5fc9879ea4927/users/a2509289eaeb48faaa83b6eba039b5c3/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.859842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01654f0190a14e7691d5fc9879ea4927, user_id=a2509289eaeb48faaa83b6eba039b5c3, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:57.862665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:57.877055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2387] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:24:57 2022] PUT /identity/v3/projects/01654f0190a14e7691d5fc9879ea4927/users/a2509289eaeb48faaa83b6eba039b5c3/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:57.887847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:57.888123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:57.888655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:57.920552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] MFA Rules not processed for user `a2509289eaeb48faaa83b6eba039b5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:57.921871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:57.993755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2388] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:58.005529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:58.007322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:58.017325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:58.086966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397edb4f-16c2-48f2-aae9-d926574dd4a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:58.092974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:58.093672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:58.093885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:58.094165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:58.103174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:58.104866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364e246-daf6-41fe-a3a5-4db63dc5fef8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:58.114187 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2389] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:58.906706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:58.908272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:58.974964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12a06a-29fd-4a2d-bb63-d30328f4e706 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:58.980781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:58.981482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:58.981755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:58.982021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] PATH_INFO: `/v3/users/97a90663fe3d49a2a426efe654a546b8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:58.983264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a90663fe3d49a2a426efe654a546b8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:58.985270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:59.025331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:59.038711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a90663fe3d49a2a426efe654a546b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.045412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a90663fe3d49a2a426efe654a546b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.054488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a90663fe3d49a2a426efe654a546b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.055150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a90663fe3d49a2a426efe654a546b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.067733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a90663fe3d49a2a426efe654a546b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.073714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2390] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:58 2022] DELETE /identity/v3/users/97a90663fe3d49a2a426efe654a546b8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:59.079669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:59.080156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:59.080296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:59.100192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:24:59.101396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:59.160863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2391] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:24:59.171145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:59.172739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:59.229442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e16ed7-9dce-4469-ae14-9ff90cda9e56 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:59.233464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:59.234055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:59.234322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:59.234590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:59.243800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:59.245249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:59.255139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2392] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:24:59.552102 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:59.554162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:24:59.569261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9e9e6-b68c-4f70-a512-3b0dad41d82e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:59.574774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:59.575481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:59.575690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:59.575953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] PATH_INFO: `/v3/projects/68f784e682994643ab5d25c50efd60a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:59.577555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f784e682994643ab5d25c50efd60a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:59.579792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:59.634488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f784e682994643ab5d25c50efd60a4', 'request_id': 'req-979d6446-c597-40e4-b97f-8d03b93af4c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.639780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f784e682994643ab5d25c50efd60a4', 'request_id': 'req-979d6446-c597-40e4-b97f-8d03b93af4c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.646177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f784e682994643ab5d25c50efd60a4', 'request_id': 'req-979d6446-c597-40e4-b97f-8d03b93af4c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.661475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] The token cache is being invalidate because project 68f784e682994643ab5d25c50efd60a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:24:59.661752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.662385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.663878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2393] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:24:59 2022] DELETE /identity/v3/projects/68f784e682994643ab5d25c50efd60a4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:24:59.669100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:24:59.679199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d415779-4bf7-458f-a3e2-33e28a88edbc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:24:59.735477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:24:59.736089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:24:59.736390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:24:59.736659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] PATH_INFO: `/v3/users/ce2ff606bb3845ffbd3a645427cd947b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:24:59.737855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2ff606bb3845ffbd3a645427cd947b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:24:59.739778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:24:59.790079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2ff606bb3845ffbd3a645427cd947b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.795401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:24:59.797606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2ff606bb3845ffbd3a645427cd947b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.807972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2ff606bb3845ffbd3a645427cd947b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.808735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2ff606bb3845ffbd3a645427cd947b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.821403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a456ab26-f9a1-4fc0-8d60-5971d5081031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2ff606bb3845ffbd3a645427cd947b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:24:59.827163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2394] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:24:59 2022] DELETE /identity/v3/users/ce2ff606bb3845ffbd3a645427cd947b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:00.116629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.118150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:00.131007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6446-c597-40e4-b97f-8d03b93af4c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.143234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:00.194817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.195788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.196323 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.196819 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] PATH_INFO: `/v3/projects/21e73f4acef24c9db1fcd57187107fb8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.198762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e73f4acef24c9db1fcd57187107fb8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.201006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.249931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e73f4acef24c9db1fcd57187107fb8', 'request_id': 'req-0a970838-8edf-44a0-9ebd-f51687fc463f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:00.255632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e73f4acef24c9db1fcd57187107fb8', 'request_id': 'req-0a970838-8edf-44a0-9ebd-f51687fc463f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:00.262007 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e73f4acef24c9db1fcd57187107fb8', 'request_id': 'req-0a970838-8edf-44a0-9ebd-f51687fc463f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:00.276425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] The token cache is being invalidate because project 21e73f4acef24c9db1fcd57187107fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:00.276705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:00.277326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a970838-8edf-44a0-9ebd-f51687fc463f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:00.278831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2395] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:00 2022] DELETE /identity/v3/projects/21e73f4acef24c9db1fcd57187107fb8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:00.283853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.284414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.284554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.306889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:00.308107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:00.369942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2396] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:00.378156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.378435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.378702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.401209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:00.402406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:00.411348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2397] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:00.415748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.425817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c9735-3340-49c5-8773-727fe9695858 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.430798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.431392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.431666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.431933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.432610 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:00.432891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.434495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.448088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2398] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:25:00.452424 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.462046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e9a6-7e1c-44e0-912b-05869e7af2d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.467157 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.467748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.468020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.468350 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.468963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.470850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.501287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2399] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:00.506962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.518741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f45b-0f9a-4e67-b072-a1ca96346bb9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.584003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.584640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.584908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.585177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.585815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.587729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.611849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2400] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:00.616820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.628972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c960f-08b1-4cf3-bf77-70b12b539043 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.634614 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.635248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.635522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.635785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.636471 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.638363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.649515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2401] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:00.653615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.664459 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae69a1-ecc0-40e8-b682-438c9894381d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.669904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.670505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.670783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.671050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.671691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.673331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736834a-fe70-474b-9e13-ef4f9e303ce1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.684291 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2402] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:00.689365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.699984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676169ce-20e7-46ba-ac8f-59608fcfde1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.705025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.705665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.705967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.706295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] PATH_INFO: `/v3/projects/46a9465197bf442796380b2e0ebd54f5/users/7157e3ea16034748805904da5bc554c6/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.721000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a9465197bf442796380b2e0ebd54f5, user_id=7157e3ea16034748805904da5bc554c6, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.723743 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.742133 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2403] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:00 2022] PUT /identity/v3/projects/46a9465197bf442796380b2e0ebd54f5/users/7157e3ea16034748805904da5bc554c6/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:00.753304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.753584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.753847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.782930 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] MFA Rules not processed for user `7157e3ea16034748805904da5bc554c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:00.784285 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:00.811790 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:00.853964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2404] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:00.859258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.908800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:00.928414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4215-f46e-4c5f-9199-0f0db1c5199b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:00.933314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:00.933917 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:00.934188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:00.934451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:00.935070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:00.937018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:00.964953 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2405] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:00.969138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:00.979836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd63dd-b0f7-4186-b43b-9195ec0c8e9d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.039267 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.039862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.040158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.040497 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.041108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.043021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.067258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2406] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:00 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:01.071227 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.081590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e565c-194c-4925-a60c-5c91230c56ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.086431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.087017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.087286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.087553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.088184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.089790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.100136 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2407] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:01.104278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.115396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de3036-efa5-4f13-a251-22c5ca58a665 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.121561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.122194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.122469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.122733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] PATH_INFO: `/v3/projects/d272979c327f46208aab4953e08b083c/users/7afbcd8f31bc4f26a276f450913292ab/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.138734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d272979c327f46208aab4953e08b083c, user_id=7afbcd8f31bc4f26a276f450913292ab, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.141447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.157923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2408] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:01 2022] PUT /identity/v3/projects/d272979c327f46208aab4953e08b083c/users/7afbcd8f31bc4f26a276f450913292ab/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:01.162972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.173370 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121eff0-6c11-4a40-a045-61cb73034865 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.231878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.232529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.232788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.233080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.233722 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.235340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ec37d-2174-4d2e-9904-f9c0b7478675 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.246096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2409] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:01.250895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.267032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c17a3e-d26f-4731-8ca6-0c16ea40b4c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.273912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.274566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.274841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.275106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] PATH_INFO: `/v3/projects/d272979c327f46208aab4953e08b083c/users/7afbcd8f31bc4f26a276f450913292ab/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.277279 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d272979c327f46208aab4953e08b083c, user_id=7afbcd8f31bc4f26a276f450913292ab, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.279942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.291419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2410] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:01 2022] PUT /identity/v3/projects/d272979c327f46208aab4953e08b083c/users/7afbcd8f31bc4f26a276f450913292ab/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:01.303230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.303510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.303736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.337787 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] MFA Rules not processed for user `7afbcd8f31bc4f26a276f450913292ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:01.338988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:01.408048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2411] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:01.419175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.421166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:01.494167 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42fb91-1390-4d96-b0e0-596bd0eab771 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.497818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.498429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.498702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.498969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.509863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.512002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.526473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:01.572835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:01.586325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f51bc3-01be-4e34-9276-6a2cd6362df6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:01.590375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:01.591003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:01.591282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:01.591575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:01.628574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:01.653282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:01.654659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00bf63-4c8c-4f1c-b389-d311d81ca4d0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:01.663994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2413] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:05.548388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:05.549984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:05.576594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:05.619993 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd250f-ba19-4c0b-8a15-d264a82097ca service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:05.624754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:05.625387 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:05.625665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:05.625943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] PATH_INFO: `/v3/users/d4804971688d4597be96394e7ffc173e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:05.627183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4804971688d4597be96394e7ffc173e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:05.629200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:05.685374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4804971688d4597be96394e7ffc173e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:05.692620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4804971688d4597be96394e7ffc173e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:05.701645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4804971688d4597be96394e7ffc173e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:05.702352 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4804971688d4597be96394e7ffc173e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:05.714849 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4804971688d4597be96394e7ffc173e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:05.720938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2414] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:05 2022] DELETE /identity/v3/users/d4804971688d4597be96394e7ffc173e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:05.725786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:05.726278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:05.726378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:05.745726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:05.746927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:05.807466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2415] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:05.817781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:05.819272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:05.882615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbfaa6-44ef-44e6-a1a3-c01f289e128e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:05.887231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:05.887837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:05.888115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:05.888402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:05.898714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:05.900150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:05.909979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2416] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:06.161043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.162858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.176655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a20a8-fd3a-42da-b0ab-5b0391bfed8a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.182748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.183537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.183862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.184159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] PATH_INFO: `/v3/users/7157e3ea16034748805904da5bc554c6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.185376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7157e3ea16034748805904da5bc554c6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.187621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.201694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.204069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.222593 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7435a3-9574-4ae9-9cf9-b5996aa4c14d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.229504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.230344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.230624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.231038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] PATH_INFO: `/v3/projects/c77d348619b2492fb4049fdd6909f416` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.232069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77d348619b2492fb4049fdd6909f416)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.234177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.250213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7157e3ea16034748805904da5bc554c6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.250886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7157e3ea16034748805904da5bc554c6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.259674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7157e3ea16034748805904da5bc554c6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.274236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7157e3ea16034748805904da5bc554c6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.274894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-acc25fe6-0c93-4fe4-b9f9-fed549935116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7157e3ea16034748805904da5bc554c6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.276997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:06.285244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77d348619b2492fb4049fdd6909f416', 'request_id': 'req-8c5accd8-6b4b-4bff-9500-ab108b827720'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.289396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2417] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/users/7157e3ea16034748805904da5bc554c6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:06.293969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.294377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.294658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.294919 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77d348619b2492fb4049fdd6909f416', 'request_id': 'req-8c5accd8-6b4b-4bff-9500-ab108b827720'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.305865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77d348619b2492fb4049fdd6909f416', 'request_id': 'req-8c5accd8-6b4b-4bff-9500-ab108b827720'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.314965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:06.316163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.317287 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] The token cache is being invalidate because project c77d348619b2492fb4049fdd6909f416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:06.317573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.318244 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.319844 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2418] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/projects/c77d348619b2492fb4049fdd6909f416 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:06.323668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.325058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.360221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:06.378042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2419] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:06.387646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5accd8-6b4b-4bff-9500-ab108b827720 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.387973 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.389434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.392369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.393004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.393281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.393839 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] PATH_INFO: `/v3/users/291d1975c452459eb1b485e045e52e80` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.395272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] RBAC: Authorizing `identity:delete_user(user_id=291d1975c452459eb1b485e045e52e80)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.397369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.447001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bba8c-8a58-43e8-8b91-94d9a0b49bb8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.448133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291d1975c452459eb1b485e045e52e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.450727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.451335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.451613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.451885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.455416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291d1975c452459eb1b485e045e52e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.461084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.462514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.464074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291d1975c452459eb1b485e045e52e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.464812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291d1975c452459eb1b485e045e52e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.472198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2420] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:06.477483 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291d1975c452459eb1b485e045e52e80'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.483714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2421] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/users/291d1975c452459eb1b485e045e52e80 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:06.782298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.784353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.811765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.814635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c86bd9-a99f-46da-87c4-9cca158a05e8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.814877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.877569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:06.892156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.892830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.893106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.893400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] PATH_INFO: `/v3/projects/46a9465197bf442796380b2e0ebd54f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.894575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a9465197bf442796380b2e0ebd54f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.896808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e5e81-1433-433e-8fa4-1d52a647622a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:06.897178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.903096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:06.903779 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.904056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.904356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] PATH_INFO: `/v3/projects/e7c66ab9925b4542be68b3d66ed68687` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:06.905493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c66ab9925b4542be68b3d66ed68687)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:06.907408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:06.955409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a9465197bf442796380b2e0ebd54f5', 'request_id': 'req-cfc19528-57d4-49ef-b7dd-42127b629c54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.956188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c66ab9925b4542be68b3d66ed68687', 'request_id': 'req-7bbcf6fc-44c1-445c-a356-4be659d6f48e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.961465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a9465197bf442796380b2e0ebd54f5', 'request_id': 'req-cfc19528-57d4-49ef-b7dd-42127b629c54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.961873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c66ab9925b4542be68b3d66ed68687', 'request_id': 'req-7bbcf6fc-44c1-445c-a356-4be659d6f48e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.970027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a9465197bf442796380b2e0ebd54f5', 'request_id': 'req-cfc19528-57d4-49ef-b7dd-42127b629c54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.971969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c66ab9925b4542be68b3d66ed68687', 'request_id': 'req-7bbcf6fc-44c1-445c-a356-4be659d6f48e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.984461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] The token cache is being invalidate because project 46a9465197bf442796380b2e0ebd54f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:06.984748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.985360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.985690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] The token cache is being invalidate because project e7c66ab9925b4542be68b3d66ed68687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:06.985979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.986614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7bbcf6fc-44c1-445c-a356-4be659d6f48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:06.986975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2422] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/projects/46a9465197bf442796380b2e0ebd54f5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:06.988121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2423] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/projects/e7c66ab9925b4542be68b3d66ed68687 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:06.990941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:06.992324 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:06.992695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69637fe1-30e1-44a0-b655-7671f2d06a73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:06.992974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69637fe1-30e1-44a0-b655-7671f2d06a73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:06.993317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69637fe1-30e1-44a0-b655-7671f2d06a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.013239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69637fe1-30e1-44a0-b655-7671f2d06a73 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:07.014444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69637fe1-30e1-44a0-b655-7671f2d06a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.018694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:07.058830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19528-57d4-49ef-b7dd-42127b629c54 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.084655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2424] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:07.088128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.088444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.088719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.107682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:07.108936 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.116854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2425] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:07.118399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.119034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.119312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.119581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] PATH_INFO: `/v3/users/7afbcd8f31bc4f26a276f450913292ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.120746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afbcd8f31bc4f26a276f450913292ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.121010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.122663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.129933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38074bd4-180f-44d0-bd1c-2c772ba43c1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.134576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.135151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.135431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.135699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.136390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:07.136692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.138282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.151277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2426] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:07.155751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.165782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fb58c-3c99-40cc-a7a1-6d23a43a4f49 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.170865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.171529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.171826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.172121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.172789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.173888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afbcd8f31bc4f26a276f450913292ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.174559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afbcd8f31bc4f26a276f450913292ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.174672 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.183147 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afbcd8f31bc4f26a276f450913292ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.196685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afbcd8f31bc4f26a276f450913292ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.197539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afbcd8f31bc4f26a276f450913292ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.204688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2427] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:07.209115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.210759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.212107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2428] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:06 2022] DELETE /identity/v3/users/7afbcd8f31bc4f26a276f450913292ab => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:07.253820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:07.286450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca584ac-103d-4118-905c-0d1ca7945202 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.347613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.348458 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.348738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.349001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.349632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.351623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.380974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2429] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:07.382339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.393372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6daa8-61a3-4ec2-90da-9ce29ffaad6b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.398838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.399389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.399808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.400149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.400957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.402546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.413570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2430] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:07.417592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.427663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e545b9-d9f0-4f1c-a1e9-e7545a9e1a19 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.432523 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.433125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.433399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.433663 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.434302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.435902 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.445995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2431] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:07.450291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.460810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0476c92-c0c1-4e40-8f44-d604918d7e8c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.466097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.467166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.467441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.467710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] PATH_INFO: `/v3/projects/d59946653edb462c9fea8d50df36a1fa/users/90e3d925e494471eb5ab1ee8aac4e800/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.485189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59946653edb462c9fea8d50df36a1fa, user_id=90e3d925e494471eb5ab1ee8aac4e800, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.488548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91333cd-83c6-4539-bf4b-6e0f7aba3023 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.503340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2432] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:07 2022] PUT /identity/v3/projects/d59946653edb462c9fea8d50df36a1fa/users/90e3d925e494471eb5ab1ee8aac4e800/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:07.511470 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.513717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.513971 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.514265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.514560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.529043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9be9c9-5959-4507-85a8-53502169fc84 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.542831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:07.548944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] MFA Rules not processed for user `90e3d925e494471eb5ab1ee8aac4e800`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:07.551215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.592510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.593468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.593740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.594009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] PATH_INFO: `/v3/projects/d272979c327f46208aab4953e08b083c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.595155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d272979c327f46208aab4953e08b083c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.597113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.616810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2433] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:07.621260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.632196 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a51c-0dc9-4b88-aca9-68fdd016d45a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.645182 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd272979c327f46208aab4953e08b083c', 'request_id': 'req-bae8ff3e-1220-4482-b4b5-2a39f14813f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.650823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd272979c327f46208aab4953e08b083c', 'request_id': 'req-bae8ff3e-1220-4482-b4b5-2a39f14813f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.654473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:07.659694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd272979c327f46208aab4953e08b083c', 'request_id': 'req-bae8ff3e-1220-4482-b4b5-2a39f14813f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.672841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] The token cache is being invalidate because project d272979c327f46208aab4953e08b083c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:07.673123 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.673772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bae8ff3e-1220-4482-b4b5-2a39f14813f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:07.675349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2434] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:07 2022] DELETE /identity/v3/projects/d272979c327f46208aab4953e08b083c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:07.680148 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8edaef-5a0a-4e4b-8b0f-3d7bafb8e16b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.680446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8edaef-5a0a-4e4b-8b0f-3d7bafb8e16b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.680814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8edaef-5a0a-4e4b-8b0f-3d7bafb8e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.690308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.690920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.691191 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.691457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.692112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.694143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.701367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2f8edaef-5a0a-4e4b-8b0f-3d7bafb8e16b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:07.702751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f8edaef-5a0a-4e4b-8b0f-3d7bafb8e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.712525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2435] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:07.720424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.720791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.721081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.724770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2436] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:07.726309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.727861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.741334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:07.742751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.813112 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65934-b845-4ea4-8a7a-eb5e1ffeaea2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.814144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2437] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:07.819224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.819224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.819224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.819224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.820029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.822095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.823700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.841622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d4ec2-8130-451e-96b9-a3e5d0795d18 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.844913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.845851 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.847437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.848542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.849192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:07.849486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.852619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.854599 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2438] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:07.858369 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.865268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2439] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:25:07.869378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.872933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298774da-5e50-45c8-a4b5-ba1f1085ec30 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.883482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904e270-715e-4da5-ba3f-c5c1c77abaf8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.885035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.885686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.885963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.886236 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.886837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.886957 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.887429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.887700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.887964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.888529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.888631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.890463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.901618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2440] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:07.902300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.904725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:07.918705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba92102-4cf0-4a33-b4c5-b64f3cdf4e5a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.920876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2441] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:07 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:07.923886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:07.925004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:07.980708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.981332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.981603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.981867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] PATH_INFO: `/v3/users/a2509289eaeb48faaa83b6eba039b5c3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.982969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2509289eaeb48faaa83b6eba039b5c3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.984936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:07.986955 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59563375-25e3-4501-ab0b-25cb289f57be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:07.991828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:07.992466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:07.992744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:07.993012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:07.993648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:07.995280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.008624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2442] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:08.009848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.021336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9a293-3133-42ea-98a9-a49c0c003f88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.027369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.027990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.028345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.028554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.029169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.031095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.034793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2509289eaeb48faaa83b6eba039b5c3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.035519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2509289eaeb48faaa83b6eba039b5c3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.044081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2509289eaeb48faaa83b6eba039b5c3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.053513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2443] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:08.054298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.055905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.060646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2509289eaeb48faaa83b6eba039b5c3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.061367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2509289eaeb48faaa83b6eba039b5c3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.068115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eae60c-9f5c-45bb-9246-48d5158caa26 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.072765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.073405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.073680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.073923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2444] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:07 2022] DELETE /identity/v3/users/a2509289eaeb48faaa83b6eba039b5c3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:08.074053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] PATH_INFO: `/v3/projects/3cb7fb832b844c20883e74a63e494b3e/users/ade8c65b068143c99746d5370840a9e8/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.074711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.076012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.089366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb7fb832b844c20883e74a63e494b3e, user_id=ade8c65b068143c99746d5370840a9e8, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.092013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820428-d612-4792-8ea5-9c405260ecdb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.105574 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2445] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:08 2022] PUT /identity/v3/projects/3cb7fb832b844c20883e74a63e494b3e/users/ade8c65b068143c99746d5370840a9e8/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:08.106852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02631bd0-36c7-4c8d-8e9b-18702b85c8d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.107130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02631bd0-36c7-4c8d-8e9b-18702b85c8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.107448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02631bd0-36c7-4c8d-8e9b-18702b85c8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.115295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:08.126905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-02631bd0-36c7-4c8d-8e9b-18702b85c8d7 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:08.128159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02631bd0-36c7-4c8d-8e9b-18702b85c8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.158451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7e9b-730a-43ab-9301-0eccda7bb2dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.189552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2446] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:08.191892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:08.199931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.200367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.200648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.222609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.223332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.223684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.223864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.224533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.226173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.228519 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] MFA Rules not processed for user `ade8c65b068143c99746d5370840a9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:08.229708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.235900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2447] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:08.236731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.238422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.298603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978576b2-7680-45b0-a7f0-27b73ee27c5f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.301515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2448] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:08.302436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.305650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.306696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.306818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.307189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.312815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e75d-8f38-41ce-9305-760e468758b3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.317508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.317665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.318220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.318492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.318753 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.318919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.319380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.321022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.328481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2449] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:08.329247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.331311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2450] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:08.335524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.338510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4f901-4655-48b1-90a7-8b61ea6aa8b7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.343495 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.344085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.344391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.344660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.345269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.346332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe6980-e677-4797-b597-02a2b392d6c7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.347378 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.351535 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.352226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.352524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.352797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] PATH_INFO: `/v3/projects/ff002869dd484950b7550bf4f51b6f11/users/d35d2ec892f549f4bc260b4e50448481/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.373228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff002869dd484950b7550bf4f51b6f11, user_id=d35d2ec892f549f4bc260b4e50448481, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.376676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8460650-ec5a-46d8-8bef-f9fe6c7173a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.377260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2451] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:08.381853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.406108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2452] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:08 2022] PUT /identity/v3/projects/ff002869dd484950b7550bf4f51b6f11/users/d35d2ec892f549f4bc260b4e50448481/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:08.409513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.410005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.410645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.443802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] MFA Rules not processed for user `d35d2ec892f549f4bc260b4e50448481`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:08.445187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.478255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823d70-98be-4208-99f8-57ab5bfa4564 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.521141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:08.546656 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2453] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:08.549478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:08.551135 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.559746 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.560866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.561387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.562086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.562764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.564683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.592391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2454] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/users => generated 295 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:08.596160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.606934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7a71-4179-4367-af0c-d8c9f2796b57 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.612343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.612945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.613209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.613544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.614632 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.616173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.626833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2455] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:08.628145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015be0-c8fe-4d0d-9ba0-966dc8c1b32e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.631351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.634503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.635186 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.635422 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.635751 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.636602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.638522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.641336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86fd1-1442-45ed-952c-d50510c7804a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.646741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.647914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.648190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.648483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] PATH_INFO: `/v3/projects/a80602e45dd24badaf478feed0c03a78/users/b8507657571849f68efe4d0fe17c7a82/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.667377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80602e45dd24badaf478feed0c03a78, user_id=b8507657571849f68efe4d0fe17c7a82, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.670906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.677224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2456] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:08.681460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.683741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.689774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2457] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:08 2022] PUT /identity/v3/projects/a80602e45dd24badaf478feed0c03a78/users/b8507657571849f68efe4d0fe17c7a82/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:08.690839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.748942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7aab8e-1e6d-495c-b4a8-e08197eaf684 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.758675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e19e-d65d-4018-9028-f129433aa4b8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.763423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.764024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.764291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.764589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.765230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.767124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.792978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2458] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:08.797179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.806213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.806968 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.807240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.807342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0735e-01f7-447c-8770-e2649c0ce82d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.807503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] PATH_INFO: `/v3/projects/01654f0190a14e7691d5fc9879ea4927` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.808662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01654f0190a14e7691d5fc9879ea4927)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.810609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.812110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.812797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.813073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.813343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.814037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.815635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.826679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2459] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:08.827480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.834631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:08.841110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f103dd-bd49-45bc-a29f-4d6e952c42e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.846596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.847190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.847462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.847729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.849029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.850626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.859945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01654f0190a14e7691d5fc9879ea4927', 'request_id': 'req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.861627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2460] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:08.862452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.869631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01654f0190a14e7691d5fc9879ea4927', 'request_id': 'req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.871919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bf7b-1c4b-43db-8fc6-9fbc591df3b7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.878380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01654f0190a14e7691d5fc9879ea4927', 'request_id': 'req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.889845 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] The token cache is being invalidate because project 01654f0190a14e7691d5fc9879ea4927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:08.890140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.890846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:08.892892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2461] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:08 2022] DELETE /identity/v3/projects/01654f0190a14e7691d5fc9879ea4927 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:08.893720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:08.895757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.934606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:08.935266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.935559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.935828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] PATH_INFO: `/v3/projects/a80602e45dd24badaf478feed0c03a78/users/b8507657571849f68efe4d0fe17c7a82/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.937806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80602e45dd24badaf478feed0c03a78, user_id=b8507657571849f68efe4d0fe17c7a82, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:08.940437 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edd3a8-1fec-4b1f-b98e-0742a28e33b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:08.951069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2462] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:08 2022] PUT /identity/v3/projects/a80602e45dd24badaf478feed0c03a78/users/b8507657571849f68efe4d0fe17c7a82/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:08.952293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7961-2fac-4f96-bb2a-767d48c1d37a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:08.952579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7961-2fac-4f96-bb2a-767d48c1d37a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:08.952848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b7961-2fac-4f96-bb2a-767d48c1d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:08.970650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8114ed-aa78-470e-b1ca-6a5bbfc672c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:08.972364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a02b7961-2fac-4f96-bb2a-767d48c1d37a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:08.974273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a02b7961-2fac-4f96-bb2a-767d48c1d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:08.983314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:09.028050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.028744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.029016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.029284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] PATH_INFO: `/v3/projects/d2ef483b4bbb44759fef752823f179be/users/0399d66ca7714222ac0433a60f8a7f42/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.037355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2463] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:09.038680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.039021 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.039348 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.044756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ef483b4bbb44759fef752823f179be, user_id=0399d66ca7714222ac0433a60f8a7f42, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.047436 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f99d5-d876-4f06-9a5f-a31dde237e10 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.063848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2464] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:08 2022] PUT /identity/v3/projects/d2ef483b4bbb44759fef752823f179be/users/0399d66ca7714222ac0433a60f8a7f42/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:09.065029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.065303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.065566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.070259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] MFA Rules not processed for user `b8507657571849f68efe4d0fe17c7a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:09.071552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.085715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:09.086884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.129271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:09.140107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2465] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:09.140990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.155514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2466] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:09.156441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.157803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.208084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b35d43-fe7b-4ad0-8633-142a1f32466b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.212843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.213453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.213727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.214127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.214775 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.216005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1e4c4-dd48-4ac2-9160-158d75c55052 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.216401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.219647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.220247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.220585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.220804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.226586 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2467] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:09.227376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.233437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.234963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.236452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3b56-d726-4dab-b22e-da30af308e30 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.241602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.242168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.242444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.242706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.243329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:09.243622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.244587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2468] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:09.245199 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.245336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.255311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706db67d-f9d4-43ef-9501-d2c38e739568 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.258527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2469] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:09.260772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.261424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.261697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.262150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] PATH_INFO: `/v3/projects/d2ef483b4bbb44759fef752823f179be/users/0399d66ca7714222ac0433a60f8a7f42/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.263152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.264682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ef483b4bbb44759fef752823f179be, user_id=0399d66ca7714222ac0433a60f8a7f42, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.267890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d605a3-0ab9-418a-8285-3efbc6dd48ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.275108 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5f49-ca15-4e71-9a3c-9dbe66e5ff8e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.284308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2470] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:09 2022] PUT /identity/v3/projects/d2ef483b4bbb44759fef752823f179be/users/0399d66ca7714222ac0433a60f8a7f42/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:09.291410 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.291692 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.291969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.295434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:09.321679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] MFA Rules not processed for user `0399d66ca7714222ac0433a60f8a7f42`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:09.322835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.335656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.336050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.336352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.336620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.337231 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.339171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.369671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2471] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:09.369671 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.371025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.393082 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2472] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:09.393971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.404934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995526b6-13a8-47f5-a994-d8b573447142 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.431850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620a086-a297-490c-b11f-2ba77a2c80c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.435812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.436482 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.436747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.437004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.438756 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.455486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:09.462954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.463577 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.463856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.464248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.464913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.466894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.490194 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2473] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:09.494098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.499388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.501569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.503541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0bda9-4a3f-474a-a71f-3aa69101253b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.507716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.508336 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.508612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.508880 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.511269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:09.512062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.523274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ac928-ffe2-45fe-890b-4e50b69d891a service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.528110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.528921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.529197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.529467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.530128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.531742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.545997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2475] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:09.548088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.561601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e25ebc-050d-4a50-b74e-4b33b1875771 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.567172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.568039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.568418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.568665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.569851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.571465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.582342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2476] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:09.582803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.585208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e62a-b5d8-4739-8bc3-a0feaaae73d3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.586553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.596716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:09.597086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50022b13-4bf1-4894-936d-63a93f869088 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.602567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.603229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.603504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.603774 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] PATH_INFO: `/v3/projects/1ea00829f46e48beb60a2517772df547/users/a85d795b2ab7498bb608d5e94d1c8442/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.621402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea00829f46e48beb60a2517772df547, user_id=a85d795b2ab7498bb608d5e94d1c8442, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.624107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.640209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2478] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:09 2022] PUT /identity/v3/projects/1ea00829f46e48beb60a2517772df547/users/a85d795b2ab7498bb608d5e94d1c8442/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:09.644639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.645003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.645818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.669238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:09.670475 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.740509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2479] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:09.755807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:09.757667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:09.780637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:09.824747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4943b8-8a67-48dd-8d0a-f565407fbbba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:09.832642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:09.832642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:09.832642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:09.832642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:09.842524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:09.844016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd67f9-011a-42cb-8d74-fad13dc93728 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:09.855502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2480] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:10.038094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.049466 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb234de-b13a-40fc-b94e-a2a835242072 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.054848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.055560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.055938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.056363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] PATH_INFO: `/v3/users/d35d2ec892f549f4bc260b4e50448481` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.057669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35d2ec892f549f4bc260b4e50448481)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.059667 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.113280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35d2ec892f549f4bc260b4e50448481'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.121188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35d2ec892f549f4bc260b4e50448481'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.130307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35d2ec892f549f4bc260b4e50448481'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.131023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35d2ec892f549f4bc260b4e50448481'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.142152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:10.144890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35d2ec892f549f4bc260b4e50448481'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.153351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2481] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:10 2022] DELETE /identity/v3/users/d35d2ec892f549f4bc260b4e50448481 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:10.158124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7eb6-c659-4da5-9393-fd175f50eebc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.158463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7eb6-c659-4da5-9393-fd175f50eebc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.158738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7eb6-c659-4da5-9393-fd175f50eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.180959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-888c7eb6-c659-4da5-9393-fd175f50eebc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:10.182186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-888c7eb6-c659-4da5-9393-fd175f50eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.258363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2482] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:10.268423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.270063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.336991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41768f-828d-4f43-b6b4-373c5821af06 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.340853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.341487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.341763 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.342027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.351376 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.352847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.362869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2483] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:10.562574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.562996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.563365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.592946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] MFA Rules not processed for user `a85d795b2ab7498bb608d5e94d1c8442`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:10.594228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.675708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.677977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.678171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2484] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:10.685355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.686841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.701448 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaebe5-4115-4d66-bc04-d01af74ef9b0 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.707342 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.708895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.709238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.709576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] PATH_INFO: `/v3/projects/ff002869dd484950b7550bf4f51b6f11` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.711828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff002869dd484950b7550bf4f51b6f11)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.714490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.715291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61378b63-065a-4d4e-8630-56e69a7f5dae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.722379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.723235 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.723581 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.724141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.724943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.727393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.761334 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2485] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:10 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:10.766019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.782454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff002869dd484950b7550bf4f51b6f11', 'request_id': 'req-c2e9c804-af28-4d89-bfec-6f4f351454e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.790393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff002869dd484950b7550bf4f51b6f11', 'request_id': 'req-c2e9c804-af28-4d89-bfec-6f4f351454e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.798578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff002869dd484950b7550bf4f51b6f11', 'request_id': 'req-c2e9c804-af28-4d89-bfec-6f4f351454e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.799478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:10.811863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] The token cache is being invalidate because project ff002869dd484950b7550bf4f51b6f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:10.812145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.812801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.814281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2486] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:10 2022] DELETE /identity/v3/projects/ff002869dd484950b7550bf4f51b6f11 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:10.818429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.819995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.834033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564fc90-9e6b-4325-b963-f56de4b54fb7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.883183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c804-af28-4d89-bfec-6f4f351454e4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.887773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.888425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.888695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.888959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] PATH_INFO: `/v3/users/0399d66ca7714222ac0433a60f8a7f42` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.889070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.889680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.889954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.890072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0399d66ca7714222ac0433a60f8a7f42)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.890223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.890833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.892001 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.892750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.916744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:10.917006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2487] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:10 2022] POST /identity/v3/users => generated 308 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:10.922523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.922877 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:10.936904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b495f-6abc-40a1-a8a0-923eb22a63cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.943255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.944066 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.944379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.944646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.945591 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:10.947213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:10.950340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0399d66ca7714222ac0433a60f8a7f42'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.958570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0399d66ca7714222ac0433a60f8a7f42'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.959057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2488] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:10.963322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:10.969323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0399d66ca7714222ac0433a60f8a7f42'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.970122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0399d66ca7714222ac0433a60f8a7f42'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.975448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b785f-92a5-48ec-bd39-d57bcd8b8d5f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:10.980944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:10.981601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:10.981873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:10.982138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] PATH_INFO: `/v3/projects/d4578a9611844633a29a7dc41750cf08/users/88fe08323af6441fae2eba8467a7b729/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:10.989077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0399d66ca7714222ac0433a60f8a7f42'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:10.995918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2489] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:10 2022] DELETE /identity/v3/users/0399d66ca7714222ac0433a60f8a7f42 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:11.001709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4578a9611844633a29a7dc41750cf08, user_id=88fe08323af6441fae2eba8467a7b729, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.004466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.019948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2490] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:10 2022] PUT /identity/v3/projects/d4578a9611844633a29a7dc41750cf08/users/88fe08323af6441fae2eba8467a7b729/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:11.025299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.026770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:11.109442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13df4-83f6-435a-9bf1-81dfd2574076 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.194892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.195533 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.196059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.196241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.197017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.198997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.211204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2491] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:11.215759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.228748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03487f8-b151-4e78-a865-44b9f3ea44f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.235233 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.236089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.236780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.237060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] PATH_INFO: `/v3/projects/d4578a9611844633a29a7dc41750cf08/users/88fe08323af6441fae2eba8467a7b729/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.239490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4578a9611844633a29a7dc41750cf08, user_id=88fe08323af6441fae2eba8467a7b729, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.242952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c308a-b213-462b-ba77-44c47939f86f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.256978 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2492] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:11 2022] PUT /identity/v3/projects/d4578a9611844633a29a7dc41750cf08/users/88fe08323af6441fae2eba8467a7b729/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:11.335986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.337908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:11.364394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a852e6-1519-4af4-8a95-6766971fe365 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.425680 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:11.446832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.447539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.447876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.448220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] PATH_INFO: `/v3/projects/d2ef483b4bbb44759fef752823f179be` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.449616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ef483b4bbb44759fef752823f179be)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.451770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.533396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ef483b4bbb44759fef752823f179be', 'request_id': 'req-be0729d9-4868-41f4-b380-99985ff6f986'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:11.539517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ef483b4bbb44759fef752823f179be', 'request_id': 'req-be0729d9-4868-41f4-b380-99985ff6f986'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:11.547742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ef483b4bbb44759fef752823f179be', 'request_id': 'req-be0729d9-4868-41f4-b380-99985ff6f986'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:11.560651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] The token cache is being invalidate because project d2ef483b4bbb44759fef752823f179be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:11.561163 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:11.561853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be0729d9-4868-41f4-b380-99985ff6f986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:11.567207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2493] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:11 2022] DELETE /identity/v3/projects/d2ef483b4bbb44759fef752823f179be => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:11.569053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.569397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.569773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.591859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:11.593214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:11.676109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2494] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:11.676619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.676913 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.677184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.699611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:11.700917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:11.713948 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2495] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:11.718963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.730058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef3041-9200-46a6-bc21-da6963c367c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.734921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.735534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.735810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.736073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.737000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:11.737139 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.739043 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.751853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2496] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:25:11.756876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.767537 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adea49-82ba-4b81-9c53-6808957393a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.772431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.773380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.773655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.773988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.775445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.777808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.807869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2497] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:11 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:11.812751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.818188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:11.826259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981a72b-6455-4b60-ae5b-675bd03b1b3e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.886289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.887293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.887704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.888084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.889086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.892298 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.930272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2498] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:11 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:11.934425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.946330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec87304-0dfb-425e-8237-7dfbd88a5f1c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:11.951179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:11.951781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:11.952048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:11.952338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:11.952977 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:11.954573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5bcf2-264d-436d-b461-296b923af70b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:11.967935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2499] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:11.969281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:11.992560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7beede-dc99-457f-b9c0-f4d4b07a936b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.006657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.006657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.006657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.006657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.007046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.007046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.008130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.008130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.008423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.016327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2500] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:12.017872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.029823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9752bb-978b-46d8-b2d2-76d5f9a3668f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.036515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.037365 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.037796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.038218 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] PATH_INFO: `/v3/projects/7d111558f593434e8f796a8eaa909597/users/4c76dfde810448399e4774d88197d051/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.044467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] MFA Rules not processed for user `88fe08323af6441fae2eba8467a7b729`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:12.045369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.054270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d111558f593434e8f796a8eaa909597, user_id=4c76dfde810448399e4774d88197d051, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.057135 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d57fd-c67a-4a3d-8ba0-bbec9756eeeb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.073525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2501] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:12 2022] PUT /identity/v3/projects/7d111558f593434e8f796a8eaa909597/users/4c76dfde810448399e4774d88197d051/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:12.084475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.084783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.085069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.125003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] MFA Rules not processed for user `4c76dfde810448399e4774d88197d051`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:12.126552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.133207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2502] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:12.211493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2503] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:12.224078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.225670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.246185 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:12.296489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda720c3-ed52-413d-aa21-c426ef341668 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.348815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.349067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.349334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.349592 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.358508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.359905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc237e-9474-416d-85b3-f5402884f076 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.369078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2504] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:12.392439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.442661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:12.459993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c6709-cf2d-46d7-a517-ee78f3358eec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.464931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.465562 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.465831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.466099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] PATH_INFO: `/v3/users/4c76dfde810448399e4774d88197d051` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.467242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c76dfde810448399e4774d88197d051)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.469160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.525169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c76dfde810448399e4774d88197d051'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.525875 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c76dfde810448399e4774d88197d051'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.536663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c76dfde810448399e4774d88197d051'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.553475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c76dfde810448399e4774d88197d051'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.554938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c76dfde810448399e4774d88197d051'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.575885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2505] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:12 2022] DELETE /identity/v3/users/4c76dfde810448399e4774d88197d051 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:12.577780 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.578499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.579205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.591193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.593211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.604745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:12.606124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.665550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac42f39-dabd-4c93-9368-00d09606552d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.670451 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.671127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.671468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.671661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] PATH_INFO: `/v3/users/90e3d925e494471eb5ab1ee8aac4e800` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.672794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e3d925e494471eb5ab1ee8aac4e800)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.674753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.682605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2506] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:12.695888 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.697565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.736984 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e3d925e494471eb5ab1ee8aac4e800'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.737656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e3d925e494471eb5ab1ee8aac4e800'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.752432 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e3d925e494471eb5ab1ee8aac4e800'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.769583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:12.771820 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc889633-f6ee-4b42-8c9f-45e38b56bfef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.772051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e3d925e494471eb5ab1ee8aac4e800'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.772955 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-414d7808-cf4c-4ea5-bf03-86505a19a49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e3d925e494471eb5ab1ee8aac4e800'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:12.777878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.778773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.779374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.779853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.782343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.792048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2507] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:12 2022] DELETE /identity/v3/users/90e3d925e494471eb5ab1ee8aac4e800 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:12.797056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.797628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.797948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.799315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.801161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.814615 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2508] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:12.818363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:12.819580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.878773 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2509] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:12.889435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:12.891195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:12.903415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcb3f4-70fe-4bde-acbc-25c2c1af8fde service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:12.923164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:12.960419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:12.961032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:12.961320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:12.961584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:12.971861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:12.973701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:12.985804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2510] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:13.238340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.239839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.255635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a1e1-772c-4533-a69d-74573f2df70e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.273943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.274608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.274879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.275214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] PATH_INFO: `/v3/projects/7d111558f593434e8f796a8eaa909597` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.276300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d111558f593434e8f796a8eaa909597)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.278707 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.345444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d111558f593434e8f796a8eaa909597', 'request_id': 'req-564ee8fa-fc81-49e7-8334-0f99cb42713e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.356489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.358120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.371569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d111558f593434e8f796a8eaa909597', 'request_id': 'req-564ee8fa-fc81-49e7-8334-0f99cb42713e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.374873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268a474-6dbc-4566-b991-51eac5a6e55f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.381736 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d111558f593434e8f796a8eaa909597', 'request_id': 'req-564ee8fa-fc81-49e7-8334-0f99cb42713e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.407954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] The token cache is being invalidate because project 7d111558f593434e8f796a8eaa909597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:13.408525 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.409524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-564ee8fa-fc81-49e7-8334-0f99cb42713e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.416481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2511] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:13 2022] DELETE /identity/v3/projects/7d111558f593434e8f796a8eaa909597 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:13.418891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.419176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.419485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.442751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:13.443965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.459380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.460208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.460516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.460787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] PATH_INFO: `/v3/projects/d59946653edb462c9fea8d50df36a1fa` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.461927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59946653edb462c9fea8d50df36a1fa)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.463862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.506126 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2512] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:13.506989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.508208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.520030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568daa2-ab74-4e4e-8408-07dbcf660f30 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.521392 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59946653edb462c9fea8d50df36a1fa', 'request_id': 'req-f49bb897-6dd1-4620-b009-22b57be0cace'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.525522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:13.528004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59946653edb462c9fea8d50df36a1fa', 'request_id': 'req-f49bb897-6dd1-4620-b009-22b57be0cace'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.536952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59946653edb462c9fea8d50df36a1fa', 'request_id': 'req-f49bb897-6dd1-4620-b009-22b57be0cace'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.548953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] The token cache is being invalidate because project d59946653edb462c9fea8d50df36a1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:13.549288 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.550022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f49bb897-6dd1-4620-b009-22b57be0cace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.551648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2513] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:13 2022] DELETE /identity/v3/projects/d59946653edb462c9fea8d50df36a1fa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:13.553085 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.553900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.554266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.573670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:13.574952 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.580374 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.581026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.581395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.581674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] PATH_INFO: `/v3/users/a85d795b2ab7498bb608d5e94d1c8442` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.583366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85d795b2ab7498bb608d5e94d1c8442)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.583843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2514] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:13.584642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.585309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.585824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.598216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e70b8c-9fba-4b61-a466-30aa11887f12 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.603152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.603774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.604038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.604304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] PATH_INFO: `/v3/users/ade8c65b068143c99746d5370840a9e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.605469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade8c65b068143c99746d5370840a9e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.607356 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.646821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85d795b2ab7498bb608d5e94d1c8442'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.648173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85d795b2ab7498bb608d5e94d1c8442'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.659438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:13.659663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85d795b2ab7498bb608d5e94d1c8442'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.666826 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade8c65b068143c99746d5370840a9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.674241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85d795b2ab7498bb608d5e94d1c8442'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.674943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85d795b2ab7498bb608d5e94d1c8442'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.675824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade8c65b068143c99746d5370840a9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.684741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade8c65b068143c99746d5370840a9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.685366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade8c65b068143c99746d5370840a9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.687959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2515] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:13 2022] DELETE /identity/v3/users/a85d795b2ab7498bb608d5e94d1c8442 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:13.688820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.690148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.701553 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade8c65b068143c99746d5370840a9e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:13.709096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2516] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:13 2022] DELETE /identity/v3/users/ade8c65b068143c99746d5370840a9e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:13.761610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26156c04-7694-4db2-98d9-a835a8f7da5a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.804363 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:13.836313 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.837046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.837326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.837602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.838351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:13.838759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.840435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.856700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2517] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:13.857791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.859584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.874394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721e15e-f201-46d3-8c76-75c6e8ccd22d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.884002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.884002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.884002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.884002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.884651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.885236 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.911540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:13.913585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:13.935252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626520e2-0ab0-4783-9c38-b62f584a46e9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:13.945861 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:13.946809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:13.947442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:13.947442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] PATH_INFO: `/v3/projects/1ea00829f46e48beb60a2517772df547` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:13.948688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea00829f46e48beb60a2517772df547)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:13.952365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2518] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:13 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:13.952540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:13.955997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.028999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea00829f46e48beb60a2517772df547', 'request_id': 'req-d0e0582f-2f11-40da-aff3-85c6f6599edf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.040913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea00829f46e48beb60a2517772df547', 'request_id': 'req-d0e0582f-2f11-40da-aff3-85c6f6599edf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.047173 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec2d13-29cd-4ee6-93c9-e8afb55fc23a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.049895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea00829f46e48beb60a2517772df547', 'request_id': 'req-d0e0582f-2f11-40da-aff3-85c6f6599edf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.068799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] The token cache is being invalidate because project 1ea00829f46e48beb60a2517772df547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:14.068948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.069553 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.071501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2519] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:13 2022] DELETE /identity/v3/projects/1ea00829f46e48beb60a2517772df547 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:14.072489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.073912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.108745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.109339 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.109750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.110023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.110649 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.112616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.140109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2520] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:13 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:14.140109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.141645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.155987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1764-79bf-4b6d-84c4-4172618a63db admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.162080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.162742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.163020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.163283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] PATH_INFO: `/v3/users/88fe08323af6441fae2eba8467a7b729` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.164639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fe08323af6441fae2eba8467a7b729)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.166605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.202308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:14.222969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fe08323af6441fae2eba8467a7b729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.230394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fe08323af6441fae2eba8467a7b729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.240073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fe08323af6441fae2eba8467a7b729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.240868 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fe08323af6441fae2eba8467a7b729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.253052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fe08323af6441fae2eba8467a7b729'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.259472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2521] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:14 2022] DELETE /identity/v3/users/88fe08323af6441fae2eba8467a7b729 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:14.262642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.262642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.278609 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0582f-2f11-40da-aff3-85c6f6599edf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.347514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.348434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.348777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.349150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] PATH_INFO: `/v3/projects/3cb7fb832b844c20883e74a63e494b3e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.350345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb7fb832b844c20883e74a63e494b3e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.352464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.353190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dbb49-17f0-4052-8554-d2cac0435d2c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.354162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:14.363745 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.365054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.365054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.365343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.366455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.368590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.379445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2522] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:14.384564 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.396474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43728121-9ec8-468f-9f7d-553bfa2666dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.414566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb7fb832b844c20883e74a63e494b3e', 'request_id': 'req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.426263 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb7fb832b844c20883e74a63e494b3e', 'request_id': 'req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.434965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb7fb832b844c20883e74a63e494b3e', 'request_id': 'req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.454430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] The token cache is being invalidate because project 3cb7fb832b844c20883e74a63e494b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:14.454855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.455629 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.458499 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2523] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:14 2022] DELETE /identity/v3/projects/3cb7fb832b844c20883e74a63e494b3e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:14.462205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.463985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.472117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.473306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.473635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.473905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.474551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.476216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.493873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2524] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:14.493873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.495933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.521302 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6e0a-c76a-4dfc-b800-9544e4fdce4f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.526821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.527648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.528464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.528464 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] PATH_INFO: `/v3/projects/d4578a9611844633a29a7dc41750cf08` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.529524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4578a9611844633a29a7dc41750cf08)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.532016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.544092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfcc46-feac-4bb3-a3d8-261e2a91b095 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.549615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.550247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.550659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.550963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] PATH_INFO: `/v3/users/b8507657571849f68efe4d0fe17c7a82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.552124 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8507657571849f68efe4d0fe17c7a82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.554201 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.572697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:14.587430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4578a9611844633a29a7dc41750cf08', 'request_id': 'req-f39156f5-12e9-4f78-9079-25574a03e5da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.595047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4578a9611844633a29a7dc41750cf08', 'request_id': 'req-f39156f5-12e9-4f78-9079-25574a03e5da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.604379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4578a9611844633a29a7dc41750cf08', 'request_id': 'req-f39156f5-12e9-4f78-9079-25574a03e5da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.612287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8507657571849f68efe4d0fe17c7a82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.613364 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8507657571849f68efe4d0fe17c7a82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.618078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] The token cache is being invalidate because project d4578a9611844633a29a7dc41750cf08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:14.618331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.619327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.621490 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2525] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:14 2022] DELETE /identity/v3/projects/d4578a9611844633a29a7dc41750cf08 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:14.622142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.623507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.627800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8507657571849f68efe4d0fe17c7a82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.644791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8507657571849f68efe4d0fe17c7a82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.645633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-08977445-441d-4e86-ae86-948154618a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8507657571849f68efe4d0fe17c7a82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:14.662487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2526] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:14 2022] DELETE /identity/v3/users/b8507657571849f68efe4d0fe17c7a82 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:14.667638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374ece-0533-4e75-98f1-bed20e280275 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.667926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374ece-0533-4e75-98f1-bed20e280275 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.668462 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77374ece-0533-4e75-98f1-bed20e280275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.696837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-77374ece-0533-4e75-98f1-bed20e280275 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:14.698355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77374ece-0533-4e75-98f1-bed20e280275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.704049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:14.719636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39156f5-12e9-4f78-9079-25574a03e5da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.784463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2527] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:14.787450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.788019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.788531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.815853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:14.817672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.822432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.823828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.824309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.824740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] PATH_INFO: `/v3/projects/a3363c71187747f39c444248b05fb14b/users/e81b71df8bcd4a41bdf9450d3ab5c57a/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.832731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2528] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:14.834980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.848710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3363c71187747f39c444248b05fb14b, user_id=e81b71df8bcd4a41bdf9450d3ab5c57a, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.849123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e398f6-2dd2-4a39-a5a4-dcba8be55f43 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.853442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880986-0b48-4310-86d6-fc04e1e34aa7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.854552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.855183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.855463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.855783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.856480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:14.856784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.858414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.873480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2529] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:14.878219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:14.887698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2530] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:14 2022] PUT /identity/v3/projects/a3363c71187747f39c444248b05fb14b/users/e81b71df8bcd4a41bdf9450d3ab5c57a/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:14.889415 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.889726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.889907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760d195-7446-4c2f-b909-1672ae545a0c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:14.890028 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.926249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] MFA Rules not processed for user `e81b71df8bcd4a41bdf9450d3ab5c57a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:14.927729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:14.954674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:14.955291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:14.955567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:14.955879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:14.956672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:14.958967 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:14.979216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:15.007013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2531] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:14 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:15.010681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.022069 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2532] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:15.049118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.052569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.101855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eeb64-7490-4673-bc41-577e3a7ddec7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.107006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.107616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.107884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.108150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.108823 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.110710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.127420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaa730-a934-4a00-a900-1b7e28323ac4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.136309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2533] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:15.145514 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.153211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:15.157675 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af29f-dc65-4f4a-bae2-a9876b0834bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.164496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.165507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.166012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.166273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.167215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.168897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.180630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2534] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:15.184184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.184717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.185019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.185294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.185563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.197327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1f046-2f65-4dea-b8ca-c48c40f09da6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.202803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.203404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.203677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.203941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.204618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.206655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.219123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2535] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:15.222949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.234851 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb669fd-4f14-4e33-836b-860d3d20c6c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.240615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.241270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.241575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.242168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] PATH_INFO: `/v3/projects/03d910e14afe461a9afe0fc2cff396da/users/35f5e2472cd14cafb2de7e6e2bf654e9/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.255750 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.257181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.259266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d910e14afe461a9afe0fc2cff396da, user_id=35f5e2472cd14cafb2de7e6e2bf654e9, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.262503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1f065-2c78-4307-99ba-1334c611c782 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.267491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2536] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:15.277684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2537] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:15 2022] PUT /identity/v3/projects/03d910e14afe461a9afe0fc2cff396da/users/35f5e2472cd14cafb2de7e6e2bf654e9/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:15.282370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.283805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.289224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.289706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.289793 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.301985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4055-371e-4848-b508-d7bbbeada53e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.330689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] MFA Rules not processed for user `35f5e2472cd14cafb2de7e6e2bf654e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:15.333058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.366668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:15.381005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.381647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.381975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.382248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] PATH_INFO: `/v3/projects/a80602e45dd24badaf478feed0c03a78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.383469 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80602e45dd24badaf478feed0c03a78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.385893 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.403408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2538] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:15.408109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.421260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dd6a1-0c00-4035-9c44-85f686db255f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.427100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.428153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.428559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.429149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.429995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.432226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.440688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80602e45dd24badaf478feed0c03a78', 'request_id': 'req-6b78c281-81e6-4dce-85a7-df1234bb2e1b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:15.449585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80602e45dd24badaf478feed0c03a78', 'request_id': 'req-6b78c281-81e6-4dce-85a7-df1234bb2e1b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:15.458858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80602e45dd24badaf478feed0c03a78', 'request_id': 'req-6b78c281-81e6-4dce-85a7-df1234bb2e1b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:15.463529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2539] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:15.468332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.476771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] The token cache is being invalidate because project a80602e45dd24badaf478feed0c03a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:15.477104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:15.477754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b78c281-81e6-4dce-85a7-df1234bb2e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:15.479366 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2540] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:15 2022] DELETE /identity/v3/projects/a80602e45dd24badaf478feed0c03a78 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:15.485265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320053ed-5937-4574-9cec-76c59cf99e3b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.485551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320053ed-5937-4574-9cec-76c59cf99e3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.485850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320053ed-5937-4574-9cec-76c59cf99e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.506882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-320053ed-5937-4574-9cec-76c59cf99e3b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:15.508504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-320053ed-5937-4574-9cec-76c59cf99e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.539889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad5b31-d14e-4465-9aef-825cb5e46638 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.549944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.550580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.550958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.551237 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.552099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.554326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.584548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2541] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:15.588251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.588742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.589031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.602840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2542] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:15.608360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.615259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.629568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4392db-6e9d-4a1e-96e4-391661a084b8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.632569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:15.633896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.634354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.635042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.635377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.635704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.636453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.638035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.643350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2543] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:15.648770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.654114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2544] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:15.655001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.659393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6418-6bb4-41ac-bbe9-393237693205 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.661367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:15.665925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de2d8-eb14-46ff-9b55-e35b7ebb7d72 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.666338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.667061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.667327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.667603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.668255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:15.668617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.670603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.674884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.675563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.675836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.676148 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] PATH_INFO: `/v3/projects/200bc88825164ba3bea4976d4803d505/users/8e77bdd07e224b00b65ddcd471e0ea89/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.682800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2545] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:25:15.687254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.693055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200bc88825164ba3bea4976d4803d505, user_id=8e77bdd07e224b00b65ddcd471e0ea89, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.695691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.697292 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de599f06-c234-4909-8485-f5c9c50bdd0a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.702072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.702723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.703027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.703269 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.703951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.705933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.710150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2546] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:15 2022] PUT /identity/v3/projects/200bc88825164ba3bea4976d4803d505/users/8e77bdd07e224b00b65ddcd471e0ea89/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:15.713944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.734233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2547] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:15.738283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.779997 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:15.780205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1723f-7088-4a01-9c48-daefc8cb5196 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.804390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cdf5-06d2-4e84-989f-4e3f28d69ea9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.809262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.809863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.810145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.810397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.811010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.812961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.835944 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2548] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:15.840020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.842944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.843551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.843822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.844084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.844990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.846694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.850800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf209d77-6234-47e8-807d-67439fb86505 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.856169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.856730 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.857006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.857271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.857434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2549] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:15.857984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.859659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.861504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.870370 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2550] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:15.873897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d8725-5ac9-475c-a277-c2f8f3822853 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.875245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.881015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.882149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.882149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.882409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] PATH_INFO: `/v3/projects/200bc88825164ba3bea4976d4803d505/users/8e77bdd07e224b00b65ddcd471e0ea89/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.884193 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] RBAC: Authorizing `identity:create_grant(project_id=200bc88825164ba3bea4976d4803d505, user_id=8e77bdd07e224b00b65ddcd471e0ea89, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.886738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6f288-664b-421c-b401-460f6ac9169e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.886864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e158133-3b62-41b1-b756-3b0aa5a9a60c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.891322 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.891906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.892179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.892473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.893132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:15.894773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:15.898016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2551] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:15 2022] PUT /identity/v3/projects/200bc88825164ba3bea4976d4803d505/users/8e77bdd07e224b00b65ddcd471e0ea89/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:15.906303 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2552] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:15.908152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922d2b-e1d9-497d-b18b-4404dd7eb421 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.908691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922d2b-e1d9-497d-b18b-4404dd7eb421 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.909718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:15.912634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922d2b-e1d9-497d-b18b-4404dd7eb421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.923774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c281217-aa67-4476-95b4-d33a9639eb7d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:15.941158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88922d2b-e1d9-497d-b18b-4404dd7eb421 None None] MFA Rules not processed for user `8e77bdd07e224b00b65ddcd471e0ea89`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:15.942676 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88922d2b-e1d9-497d-b18b-4404dd7eb421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:15.982057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:15.982712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:15.982984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:15.983242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] PATH_INFO: `/v3/projects/1d1df0bf099045dcb88142cbc4d31c07/users/96590ab60e5e4583981915df7d0d0ba7/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:15.998146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1df0bf099045dcb88142cbc4d31c07, user_id=96590ab60e5e4583981915df7d0d0ba7, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.000915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.003286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:16.013901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2553] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:16.019063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2554] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:15 2022] PUT /identity/v3/projects/1d1df0bf099045dcb88142cbc4d31c07/users/96590ab60e5e4583981915df7d0d0ba7/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:16.025874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.027849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.027849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.028645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.030314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:16.049195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:16.050533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:16.093047 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57b21c-e814-4422-9b0e-ea45f3dec622 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.098697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.098697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.098933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.099153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.116395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2555] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:16.126824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.128252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:16.150538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:16.170308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.171165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.181538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2556] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:16.196173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fd7e0-eab3-49de-9761-6d32be41af80 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.199972 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.200730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.201081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.201414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.213724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.215076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.225251 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2557] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:16.539141 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.548563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf79bd-bb02-4d92-b0da-d21e79099e56 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.553884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e77bdd07e224b00b65ddcd471e0ea89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '200bc88825164ba3bea4976d4803d505', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.554515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.554791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.555108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.555859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:16.556192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.557824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.571035 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2558] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:16.575091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.585966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188ab0c-db62-489e-95e8-33a9d1042098 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.590498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e77bdd07e224b00b65ddcd471e0ea89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '200bc88825164ba3bea4976d4803d505', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.591101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.591379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.591647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.592282 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.594192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.628129 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2559] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:16 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:16.629758 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.640347 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a688d-5a25-4d35-a0d9-8f819abd2693 tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.720485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e77bdd07e224b00b65ddcd471e0ea89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '200bc88825164ba3bea4976d4803d505', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.721109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.721385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.721664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.722300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:16.722599 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.724507 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.741717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2560] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:16.782062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:16.795886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a82b6-9fbe-4ad0-9a0b-5dde58f7c8ff tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:16.800692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e77bdd07e224b00b65ddcd471e0ea89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '200bc88825164ba3bea4976d4803d505', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:16.801620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:16.802176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:16.802520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] PATH_INFO: `/v3/projects/762e67ae59b94c49992f94e7d6715c46` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:16.803741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorizing `identity:delete_project(project_id=762e67ae59b94c49992f94e7d6715c46)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:16.806339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:16.867871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762e67ae59b94c49992f94e7d6715c46', 'request_id': 'req-ddf96435-81d1-48be-af40-c7c119d7392c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:16.876913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762e67ae59b94c49992f94e7d6715c46', 'request_id': 'req-ddf96435-81d1-48be-af40-c7c119d7392c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:16.884330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762e67ae59b94c49992f94e7d6715c46', 'request_id': 'req-ddf96435-81d1-48be-af40-c7c119d7392c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:16.894464 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:16.899040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] The token cache is being invalidate because project 762e67ae59b94c49992f94e7d6715c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:16.899441 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:16.900627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddf96435-81d1-48be-af40-c7c119d7392c tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:16.902627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2561] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:16 2022] DELETE /identity/v3/projects/762e67ae59b94c49992f94e7d6715c46 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:17.160644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.163814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:17.241632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4963f-674a-4245-94af-910b063948da tempest-VolumeQuotaClassesTest-447986196 tempest-VolumeQuotaClassesTest-447986196-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.319675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.320460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.321067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.321067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] PATH_INFO: `/v3/users/35f5e2472cd14cafb2de7e6e2bf654e9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.322655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f5e2472cd14cafb2de7e6e2bf654e9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.324774 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.338478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:17.365663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.365663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.365987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.418527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] MFA Rules not processed for user `96590ab60e5e4583981915df7d0d0ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:17.423733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:17.424760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f5e2472cd14cafb2de7e6e2bf654e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:17.439250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f5e2472cd14cafb2de7e6e2bf654e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:17.462314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f5e2472cd14cafb2de7e6e2bf654e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:17.463781 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f5e2472cd14cafb2de7e6e2bf654e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:17.497132 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-04ed3ef3-12ba-4f5c-8a8e-34ff2a20ceb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f5e2472cd14cafb2de7e6e2bf654e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:17.511987 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2562] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:17 2022] DELETE /identity/v3/users/35f5e2472cd14cafb2de7e6e2bf654e9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:17.522689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.523319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.523585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.560981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:17.562203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:17.569807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2563] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:17.574643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.576152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:17.591118 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947bff2-ed0c-4d19-9cca-1c5dcd6dd6d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.646435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2564] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:17.656678 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.658408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:17.662217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.662921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.663264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.663601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.664402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.666517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.700239 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2565] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:17 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:17.704477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.741209 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ece4-d149-4f7e-aebc-5f085bbd2c62 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.786916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:17.789289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d856-011f-4bdc-86e6-9376df069e5b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.798160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.798492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.799016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.799232 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.799342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.799521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.799650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.799824 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.800159 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.802188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.810608 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.812401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.827686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2566] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:17.829263 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2567] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:17 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:17.833100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.844229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b3293-967e-47f6-ba97-5d28e0194f99 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.849543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.850126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.850404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.850765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.851407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.853108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.865052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2568] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:17.869031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.876029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:17.882469 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b63902-3eb1-4d98-aa79-a6418bc9f1cf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:17.890075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:17.890920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:17.891479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:17.891761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] PATH_INFO: `/v3/projects/7008ec18676444fa9e472406eaf9ac61/users/4ba1103d93c34a3ca2f8dffd16e30502/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:17.913322 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7008ec18676444fa9e472406eaf9ac61, user_id=4ba1103d93c34a3ca2f8dffd16e30502, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:17.916663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:17.942572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:17.946698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2569] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:17 2022] PUT /identity/v3/projects/7008ec18676444fa9e472406eaf9ac61/users/4ba1103d93c34a3ca2f8dffd16e30502/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:17.956179 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a34ed-814b-43e4-9191-ed42c5c3d451 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:18.045933 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:18.046550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:18.046823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:18.047096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:18.047743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:18.049397 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:18.065176 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:18.066026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2570] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:18.075799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aac9-6c2e-4693-b05e-950bd1f63410 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:18.081791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:18.082509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:18.083438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:18.083438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] PATH_INFO: `/v3/projects/7008ec18676444fa9e472406eaf9ac61/users/4ba1103d93c34a3ca2f8dffd16e30502/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:18.085489 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7008ec18676444fa9e472406eaf9ac61, user_id=4ba1103d93c34a3ca2f8dffd16e30502, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:18.087821 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:18.102810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2571] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:18 2022] PUT /identity/v3/projects/7008ec18676444fa9e472406eaf9ac61/users/4ba1103d93c34a3ca2f8dffd16e30502/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:18.167894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:18.169685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:18.182905 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141aaf2c-e902-4de3-bed1-af553983e5ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:18.257095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:18.257759 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:18.258224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:18.258754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] PATH_INFO: `/v3/projects/03d910e14afe461a9afe0fc2cff396da` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:18.260833 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d910e14afe461a9afe0fc2cff396da)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:18.263471 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:18.333720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d910e14afe461a9afe0fc2cff396da', 'request_id': 'req-01106664-9af3-4247-8a28-71fdb603f127'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.340970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d910e14afe461a9afe0fc2cff396da', 'request_id': 'req-01106664-9af3-4247-8a28-71fdb603f127'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.354258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d910e14afe461a9afe0fc2cff396da', 'request_id': 'req-01106664-9af3-4247-8a28-71fdb603f127'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.369766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] The token cache is being invalidate because project 03d910e14afe461a9afe0fc2cff396da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:18.370142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.370982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01106664-9af3-4247-8a28-71fdb603f127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.373371 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2572] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:18 2022] DELETE /identity/v3/projects/03d910e14afe461a9afe0fc2cff396da => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:18.377234 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:18.379160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:18.394962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd0ec9-4755-4ce3-8bae-1f25b0071ca1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:18.468576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:18.469414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:18.469690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:18.470109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] PATH_INFO: `/v3/users/8e77bdd07e224b00b65ddcd471e0ea89` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:18.471881 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e77bdd07e224b00b65ddcd471e0ea89)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:18.474873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:18.545701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e77bdd07e224b00b65ddcd471e0ea89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.546685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e77bdd07e224b00b65ddcd471e0ea89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.561402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e77bdd07e224b00b65ddcd471e0ea89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.576297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e77bdd07e224b00b65ddcd471e0ea89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.577309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e77bdd07e224b00b65ddcd471e0ea89'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:18.596606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2573] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:18 2022] DELETE /identity/v3/users/8e77bdd07e224b00b65ddcd471e0ea89 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:18.942133 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:18.942720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:18.942720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:18.952083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:18.989388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] MFA Rules not processed for user `4ba1103d93c34a3ca2f8dffd16e30502`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:18.990613 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.033826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.035490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.083183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2574] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:19.097309 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.097935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:19.100697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.116417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79094710-f8f5-4e02-93ee-bafe916eee2b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.124641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.125529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.125804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.126131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] PATH_INFO: `/v3/projects/200bc88825164ba3bea4976d4803d505` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.127442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200bc88825164ba3bea4976d4803d505)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.129685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.177221 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a332b-ffd9-43af-91a3-c95847fa3177 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.186596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200bc88825164ba3bea4976d4803d505', 'request_id': 'req-133acecc-5b94-408d-86c8-c3c6c68d13a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:19.192970 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200bc88825164ba3bea4976d4803d505', 'request_id': 'req-133acecc-5b94-408d-86c8-c3c6c68d13a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:19.199910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200bc88825164ba3bea4976d4803d505', 'request_id': 'req-133acecc-5b94-408d-86c8-c3c6c68d13a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:19.215467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] The token cache is being invalidate because project 200bc88825164ba3bea4976d4803d505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:19.215779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:19.216601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-133acecc-5b94-408d-86c8-c3c6c68d13a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:19.218198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2575] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:19 2022] DELETE /identity/v3/projects/200bc88825164ba3bea4976d4803d505 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:19.223442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f37cd-aac2-45a2-a382-65af47db5dac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.223805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f37cd-aac2-45a2-a382-65af47db5dac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.224190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f37cd-aac2-45a2-a382-65af47db5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.240841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.241665 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.241916 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.242250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.244072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.246383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fc5f37cd-aac2-45a2-a382-65af47db5dac None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:19.247755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc5f37cd-aac2-45a2-a382-65af47db5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.310704 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.312570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.314481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2576] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:19.317323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:19.323798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.324083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.324389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.324527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:19.344113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:19.345387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.357922 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2578] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:19.362473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.373152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98666ea1-9ffe-4ca4-8504-1de405f52efe service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.377836 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.378465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.378738 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.379023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.379691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:19.380031 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.381617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.393967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2579] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:19.399013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.408947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9d411-aa9c-4713-b242-66b4e2e53771 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.414068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.414668 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.414964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.415226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.415859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.417788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.449598 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2580] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:19.450675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.461046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3141-ac25-470f-9aab-81733e533de3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.521785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.522479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.523216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.523629 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.524390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.526424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.548624 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2581] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:19.553413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.591578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178ea08-e4bb-4f1c-8411-4b2e8e24910f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.599190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.600057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.600314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.600635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.601293 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.603481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.615710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2582] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:19.619792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.633208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba3c6c-75d9-4caa-bbdc-c06c5a00a299 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.640713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.641419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.641998 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.642114 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.643712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.645816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0499815-f207-4223-b4a0-e81ffd6c2673 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.663632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2583] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:19.668168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.678461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba24113-7551-4d3e-a152-f0f9c6be0cbb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.683008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.683673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.683912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.684178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] PATH_INFO: `/v3/projects/b445d75ef13c4fab9705fbcf7043b8a9/users/f0c0994bb435479e9af3d13e20762d58/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.699942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b445d75ef13c4fab9705fbcf7043b8a9, user_id=f0c0994bb435479e9af3d13e20762d58, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.702385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.718637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2584] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:19 2022] PUT /identity/v3/projects/b445d75ef13c4fab9705fbcf7043b8a9/users/f0c0994bb435479e9af3d13e20762d58/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:19.730544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.730827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.731098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.761092 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] MFA Rules not processed for user `f0c0994bb435479e9af3d13e20762d58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:19.762383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:19.832071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2585] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:19.836757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.850510 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:19.902745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea3ad-e35e-4c4a-896f-703cce74cff6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:19.907725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:19.913827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:19.913827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:19.913827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:19.913827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:19.913827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:19.947943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2586] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:19.952255 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:19.963716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84127166-0994-42e4-95f5-7cfe77087cd9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.016700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:20.027815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.028467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.028864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.029060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.029777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.031815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.060214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2587] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:19 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:20.061426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.072438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe86c-caa6-4532-abeb-0451577f550b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.077402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.078029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.078259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.078645 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.079292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.080907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.095460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2588] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:20.096712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.109764 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5892927-54fe-4c23-b577-aedea8d57ecc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.115135 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.115756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.116014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.116294 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.116945 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.118524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca693-642a-4077-93de-5013f19e922e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.131424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2589] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:20.135771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.146389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60ab7f-60b8-4e83-8473-cdc91cae74ea admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.152342 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.153039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.153309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.153578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] PATH_INFO: `/v3/projects/718249b5d48f4a2ea6f85073161d5dcb/users/09da590998cb4cb2939f70ceb3b0a411/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.169800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=718249b5d48f4a2ea6f85073161d5dcb, user_id=09da590998cb4cb2939f70ceb3b0a411, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.173267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.190020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2590] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:20 2022] PUT /identity/v3/projects/718249b5d48f4a2ea6f85073161d5dcb/users/09da590998cb4cb2939f70ceb3b0a411/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:20.201064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.201334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.201604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.234711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] MFA Rules not processed for user `09da590998cb4cb2939f70ceb3b0a411`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:20.235956 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:20.304873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2591] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:20.309492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.374670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03497b6-7354-44d1-a141-4d26bb4a6e7a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.379776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.380415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.380882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.382226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.383418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.385345 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.420299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2592] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:20 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:20.422493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.435003 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ea343-7b45-4622-87ba-63f09e565eed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.495270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.495877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.496144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.496435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.497067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.499097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.524906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2593] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:20 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:20.529178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.544411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba901-c23b-4033-b9b0-5e3b05b07028 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.548123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.548754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.549024 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.549287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.549919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.551840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.563841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2594] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:20.568841 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.578796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2067208-bea2-4973-bd49-f43dcd80f6fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.584013 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.584710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.584981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.585251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] PATH_INFO: `/v3/projects/956689d880dd482a9858df0fef255abb/users/e1979f0c345c44b2afaf2d146d8a78af/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.601994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=956689d880dd482a9858df0fef255abb, user_id=e1979f0c345c44b2afaf2d146d8a78af, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.604814 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.622125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2595] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:20 2022] PUT /identity/v3/projects/956689d880dd482a9858df0fef255abb/users/e1979f0c345c44b2afaf2d146d8a78af/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:20.626314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.637029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc4e4e-a597-4ddd-ba80-b9c5fa2ceabf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.678533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:20.707205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.707909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.708209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.708479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.709113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.710673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.713861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.715705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:20.724519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2596] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:20.725563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.736493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632a23a-c632-4037-9833-9f1b285b2517 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.741847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.742575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.742887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.743153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] PATH_INFO: `/v3/projects/956689d880dd482a9858df0fef255abb/users/e1979f0c345c44b2afaf2d146d8a78af/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.745559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956689d880dd482a9858df0fef255abb, user_id=e1979f0c345c44b2afaf2d146d8a78af, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.748500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477b8ee-3ee0-4f2b-886d-265c0bbb1629 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.750584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:20.760508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2597] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:20 2022] PUT /identity/v3/projects/956689d880dd482a9858df0fef255abb/users/e1979f0c345c44b2afaf2d146d8a78af/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:20.783096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1862-8796-4b09-8e49-8c2bf6fc1ffb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.832822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.833107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.833361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.842176 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.842796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.843065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.843394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.845127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:20.864438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] MFA Rules not processed for user `e1979f0c345c44b2afaf2d146d8a78af`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:20.867264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:20.932200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.934023 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:20.945465 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:20.949706 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2599] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:20.962643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:20.964235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:20.976059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b418d5-6743-456f-a82b-8a924283eab6 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:20.979964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:20.980712 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:20.981083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:20.981447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:20.992511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:20.994077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:21.006664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2600] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:21.559961 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:21.570377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd10c35-5353-47f6-ace5-9ae68301afdb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:21.579995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:21.580750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:21.581092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:21.581531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:21.643086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:21.644548 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:21.655718 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2601] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:22.350380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:22.352002 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:22.414041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9f49f-055d-4d47-87ec-d8a17fb1fd3b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:22.418332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:22.419089 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:22.419331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:22.419587 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:22.421446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:22.457859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:22.491865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:22.493424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cf8e7-048b-47c1-88b7-d05b2554c65b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:22.503263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:22.872258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:22.874059 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:22.945060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:22.980060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae35a8-b714-4196-b64d-93b6990fbcd1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:22.984656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:22.985296 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:22.985620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:22.985841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] PATH_INFO: `/v3/users/e81b71df8bcd4a41bdf9450d3ab5c57a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:22.987080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81b71df8bcd4a41bdf9450d3ab5c57a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:22.989508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:23.059449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81b71df8bcd4a41bdf9450d3ab5c57a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:23.060122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81b71df8bcd4a41bdf9450d3ab5c57a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:23.071200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81b71df8bcd4a41bdf9450d3ab5c57a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:23.089610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81b71df8bcd4a41bdf9450d3ab5c57a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:23.091032 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81b71df8bcd4a41bdf9450d3ab5c57a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:23.110729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2603] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:22 2022] DELETE /identity/v3/users/e81b71df8bcd4a41bdf9450d3ab5c57a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:23.110878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:23.112967 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:23.114144 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:23.144061 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:23.145349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:23.236321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2604] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:23.249430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:23.253395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:23.314415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd6f33-4be8-4bd4-995b-d667845ca9cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:23.318680 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:23.319285 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:23.319618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:23.319883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:23.329569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:23.331357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:23.341766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2605] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:24.108957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.122280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb2f20-572e-4f9b-9d27-a7a793c1bd3a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.131950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.133617 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:24.137097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.138213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.139010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.139462 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.141798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:24.152804 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5ab69-5ff3-43a1-b40e-623bde215529 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.156242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.157984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6697b-9e2d-4038-8e35-e070af10ad1a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.158460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.159081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.159351 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.159616 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] PATH_INFO: `/v3/projects/a3363c71187747f39c444248b05fb14b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.161030 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3363c71187747f39c444248b05fb14b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.163200 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.168544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2606] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:24.235111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3363c71187747f39c444248b05fb14b', 'request_id': 'req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:24.241182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3363c71187747f39c444248b05fb14b', 'request_id': 'req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:24.244387 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:24.250346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3363c71187747f39c444248b05fb14b', 'request_id': 'req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:24.271169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] The token cache is being invalidate because project a3363c71187747f39c444248b05fb14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:24.272016 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:24.272445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-39cbc9e6-4462-4d42-93c0-d5d955e3daeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:24.274842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2607] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:24 2022] DELETE /identity/v3/projects/a3363c71187747f39c444248b05fb14b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:24.280425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.280710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.281007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.323881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:24.325403 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:24.418793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2608] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:24.427907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.428214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.428759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.449752 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:24.451242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:24.463189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2609] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:24.468007 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.483618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a719d93-c7e7-4fda-8227-8647b742daf9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.489669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.490320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.490594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.490859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.491529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:24.491817 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.493444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.509038 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2610] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:24.513838 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.525969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7143-7438-4727-ae24-ceee3fd5422e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.532730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.533413 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.533761 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.534012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.534713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.538056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.597089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2611] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:24 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:24.601350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.613110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7e05e-a0d7-4656-baf1-0b14afffacfd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.688364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:24.709444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.710080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.710358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.710628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.711282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.713661 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.741778 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2612] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:24 2022] POST /identity/v3/users => generated 282 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:24.745841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.758583 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c1958-965d-42e6-952f-c8694c81da03 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.764248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.764507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.764744 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.765009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.765651 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.767304 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.780622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2613] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:24.784805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.799877 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd57e5d-10da-4e2f-a613-6e6cff93745a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.804938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.805542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.805812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.806073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.806737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.808424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0e2e1-5f8c-4ee7-9f3e-c92bdcfbbb60 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.822971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2614] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:24.825508 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:24.838343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63326d-225a-4ef9-9b87-f5701ce1d5d7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:24.843140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:24.843799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.844104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.844416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] PATH_INFO: `/v3/projects/0cc911d8ab344a06a7c8f6034bbdfbcb/users/1b248640a229456e872aaab669843ec9/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.859830 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc911d8ab344a06a7c8f6034bbdfbcb, user_id=1b248640a229456e872aaab669843ec9, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:24.862523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:24.881347 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2615] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:24 2022] PUT /identity/v3/projects/0cc911d8ab344a06a7c8f6034bbdfbcb/users/1b248640a229456e872aaab669843ec9/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:24.891714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:24.892067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:24.892343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:24.937281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] MFA Rules not processed for user `1b248640a229456e872aaab669843ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:24.939027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:25.011812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2616] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:25.023275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:25.025438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:25.096042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99361be-50cb-47ac-85e6-d9afcbdda1a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:25.100033 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:25.100728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:25.100998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:25.101258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:25.111138 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:25.112872 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:25.125178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2617] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:25.952365 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:25.954050 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:25.976248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f5c7-b0b9-45db-83c7-9b3fbd56f592 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:25.980528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:25.981351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:25.981788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:25.982057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:25.983978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:26.057030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:26.066099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:26.077062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2618] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:28.248942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:28.251442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:28.460908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefdba9-face-4f97-ba93-ed3387aca0d0 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:28.469687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:28.472686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:28.473949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:28.474427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:28.479928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:28.641088 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:28.694580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:28.696529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90666bd-fa4c-460c-bda9-70dd4133a4a3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:28.722769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2619] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:28.997749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:28.999207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.013379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a637b-a3d8-41ef-9dcc-39ee798460e7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:29.017819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:29.018445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:29.018717 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:29.018978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] PATH_INFO: `/v3/users/f0c0994bb435479e9af3d13e20762d58` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:29.020384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c0994bb435479e9af3d13e20762d58)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:29.022498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:29.098159 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c0994bb435479e9af3d13e20762d58'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.100130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:29.105901 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c0994bb435479e9af3d13e20762d58'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.114600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c0994bb435479e9af3d13e20762d58'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.115258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c0994bb435479e9af3d13e20762d58'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.127155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c0994bb435479e9af3d13e20762d58'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.133514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2620] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:28 2022] DELETE /identity/v3/users/f0c0994bb435479e9af3d13e20762d58 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:29.138267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:29.138565 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:29.138837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:29.161064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:29.162294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.183189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:29.185343 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.258269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2621] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:29.266608 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:29.268019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.294073 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c8997-0023-42b2-8757-016a543085fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:29.302557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:29.303382 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:29.303659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:29.303935 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] PATH_INFO: `/v3/users/2e6543b377f14949a6919af4e4e187b7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:29.305348 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6543b377f14949a6919af4e4e187b7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:29.308217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:29.337087 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e61081-9c59-492f-9d27-3baf3fb46268 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:29.347217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:29.348151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:29.348725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:29.349300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:29.362735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:29.364240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:29.375406 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2622] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:29.376676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6543b377f14949a6919af4e4e187b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.385127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6543b377f14949a6919af4e4e187b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.400852 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6543b377f14949a6919af4e4e187b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.401859 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6543b377f14949a6919af4e4e187b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.416612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6543b377f14949a6919af4e4e187b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.426454 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2623] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:29 2022] DELETE /identity/v3/users/2e6543b377f14949a6919af4e4e187b7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:29.735266 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:29.737346 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.765669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c2cb8-fd3f-480b-a615-21f0d9f4d6e5 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:29.809669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:29.811284 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:29.854052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:29.854724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:29.855152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:29.855339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] PATH_INFO: `/v3/projects/b445d75ef13c4fab9705fbcf7043b8a9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:29.856570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b445d75ef13c4fab9705fbcf7043b8a9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:29.859312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:29.896411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:29.929735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b445d75ef13c4fab9705fbcf7043b8a9', 'request_id': 'req-db22d41b-c0db-415c-b4d8-12080f9b7e59'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.935945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b445d75ef13c4fab9705fbcf7043b8a9', 'request_id': 'req-db22d41b-c0db-415c-b4d8-12080f9b7e59'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.943884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b445d75ef13c4fab9705fbcf7043b8a9', 'request_id': 'req-db22d41b-c0db-415c-b4d8-12080f9b7e59'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.949962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:29.966127 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] The token cache is being invalidate because project b445d75ef13c4fab9705fbcf7043b8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:29.967909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.968824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:29.975215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2624] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:29 2022] DELETE /identity/v3/projects/b445d75ef13c4fab9705fbcf7043b8a9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:29.976837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7d7e8-a596-4aaf-a241-454f8f4c36aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:29.988824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:29.995759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:30.060645 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.061648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.062298 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.062578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] PATH_INFO: `/v3/projects/f938fbc477f34eedb17c9ead3084a207` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.064098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f938fbc477f34eedb17c9ead3084a207)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.066377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.081047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22d41b-c0db-415c-b4d8-12080f9b7e59 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.086855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.087498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.087963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.088259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] PATH_INFO: `/v3/users/09da590998cb4cb2939f70ceb3b0a411` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.089679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09da590998cb4cb2939f70ceb3b0a411)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.091427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.150604 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f938fbc477f34eedb17c9ead3084a207', 'request_id': 'req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.154878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09da590998cb4cb2939f70ceb3b0a411'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.155781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09da590998cb4cb2939f70ceb3b0a411'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.157515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f938fbc477f34eedb17c9ead3084a207', 'request_id': 'req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.166703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09da590998cb4cb2939f70ceb3b0a411'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.174567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f938fbc477f34eedb17c9ead3084a207', 'request_id': 'req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.181590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09da590998cb4cb2939f70ceb3b0a411'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.182454 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bb84174-e253-48cc-9172-d8b154e03590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09da590998cb4cb2939f70ceb3b0a411'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.193657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] The token cache is being invalidate because project f938fbc477f34eedb17c9ead3084a207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:30.193922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.195320 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62258ca-0d85-4bd4-ab68-c77ed8eff80c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.197691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2625] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:29 2022] DELETE /identity/v3/projects/f938fbc477f34eedb17c9ead3084a207 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:30.198962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2626] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:29 2022] DELETE /identity/v3/users/09da590998cb4cb2939f70ceb3b0a411 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:30.203150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.203428 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.203698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.231908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:30.233566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:30.324959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2627] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:30.328158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.329597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.329994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.367855 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:30.369433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:30.391561 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2628] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:30.393716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.412281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620324c-fcb5-48cb-8747-4c421936db79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.427819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.428670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.428950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.429214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.429886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:30.430563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.431804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.459444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2629] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:30.460109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.471450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033e79a-f4e6-4734-9d91-13bb06788017 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.476730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.477336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.477613 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.477883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.478660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.480634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.515061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2630] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:30 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:30.516642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.530939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf310-6e5c-4c00-a1de-7ee455a410ba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.540358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:30.560908 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.563791 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:30.569840 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:30.629087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.629842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.630316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.630578 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.631338 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.633630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.673138 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2631] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:30 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:30.674472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.677131 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9b389-bc89-497f-a96c-28fbc62e846d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.682963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.683882 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.684524 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.684986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] PATH_INFO: `/v3/projects/718249b5d48f4a2ea6f85073161d5dcb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.686626 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=718249b5d48f4a2ea6f85073161d5dcb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.687273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18482-011e-41b3-9be4-e28d4b375e41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.689276 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.693029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.693580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.693850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.694117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.695234 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.697617 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.716447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2632] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:30.719137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.738484 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6832450-f26e-47f1-97e5-9e63bf6f8575 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.739769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.740501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.741157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.741516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.742237 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.744503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.762982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2633] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:30.763534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718249b5d48f4a2ea6f85073161d5dcb', 'request_id': 'req-578b62de-7534-4f25-8671-cf4ea28b0cef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.767213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.779107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718249b5d48f4a2ea6f85073161d5dcb', 'request_id': 'req-578b62de-7534-4f25-8671-cf4ea28b0cef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.779414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347cbfe-f291-468e-85dd-bdba41ecec5f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.795589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718249b5d48f4a2ea6f85073161d5dcb', 'request_id': 'req-578b62de-7534-4f25-8671-cf4ea28b0cef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.825472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] The token cache is being invalidate because project 718249b5d48f4a2ea6f85073161d5dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:30.826051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.827335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:30.829931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2634] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:30 2022] DELETE /identity/v3/projects/718249b5d48f4a2ea6f85073161d5dcb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:30.834335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:30.845695 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:30.896473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:30.897372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.897650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.898405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] PATH_INFO: `/v3/projects/c3cc2647f5a743d381a29e6374edc23d/users/0d78d75c8082439d9d99fccbf9da61e1/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:30.938118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cc2647f5a743d381a29e6374edc23d, user_id=0d78d75c8082439d9d99fccbf9da61e1, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:30.944476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e3dde-c11a-452c-a5d7-5bf7bef9bd50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:30.974476 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2635] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:30 2022] PUT /identity/v3/projects/c3cc2647f5a743d381a29e6374edc23d/users/0d78d75c8082439d9d99fccbf9da61e1/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:30.975336 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b62de-7534-4f25-8671-cf4ea28b0cef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:30.985734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:30.986211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:30.987233 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.052203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] MFA Rules not processed for user `0d78d75c8082439d9d99fccbf9da61e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:31.053473 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:31.059375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:31.092563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.093660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.094532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.095299 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] PATH_INFO: `/v3/users/e1979f0c345c44b2afaf2d146d8a78af` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.097082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1979f0c345c44b2afaf2d146d8a78af)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.099664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.155008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:31.182280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1979f0c345c44b2afaf2d146d8a78af'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.183099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1979f0c345c44b2afaf2d146d8a78af'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.195209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1979f0c345c44b2afaf2d146d8a78af'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.200136 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2636] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:31.201326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.204800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:31.218551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1979f0c345c44b2afaf2d146d8a78af'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.219496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1979f0c345c44b2afaf2d146d8a78af'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.221777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782fc5a-bf5a-4e85-9fdd-bf2a01786483 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.228477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.229832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.230373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.230783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.231695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.233922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.234865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2637] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:30 2022] DELETE /identity/v3/users/e1979f0c345c44b2afaf2d146d8a78af => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:31.300799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2638] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:31 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:31.304974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.306853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:31.418002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520465c-7bdb-4a91-88fe-fc94c64bb61e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.547026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.548255 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.548702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.549106 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.552040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.556213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.586122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:31.602084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2639] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:31 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:31.603434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.614429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1737e-4dfa-4b3b-8830-80ab206b0c74 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.619688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.620616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.621100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.621566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.622456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.624332 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.640306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2640] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:31.642465 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.654727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1d74-e241-45d2-bad3-560465da2c39 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.659652 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.660373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.660938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.661341 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] PATH_INFO: `/v3/projects/6c9888492d5a448199f49e491de5fc87/users/50fe6c5b8b924dbeb64be69480203fec/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.681479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9888492d5a448199f49e491de5fc87, user_id=50fe6c5b8b924dbeb64be69480203fec, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.684557 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.702658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.704964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:31.711890 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2641] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:31 2022] PUT /identity/v3/projects/6c9888492d5a448199f49e491de5fc87/users/50fe6c5b8b924dbeb64be69480203fec/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:31.717242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.725147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c7859-59d9-4420-8b2f-b877bb8d18a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.809630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.810169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.810455 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.810733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] PATH_INFO: `/v3/projects/956689d880dd482a9858df0fef255abb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.812876 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956689d880dd482a9858df0fef255abb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.815953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.816146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67e234-0768-4161-a0e7-cda815b2d39b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.821584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.822269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.822545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.822812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.823491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.825134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.840480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2642] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:31.841621 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:31.852963 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3c22-20f6-4d59-ad97-b4008259cd81 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:31.859717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:31.860532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.860809 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.861162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] PATH_INFO: `/v3/projects/6c9888492d5a448199f49e491de5fc87/users/50fe6c5b8b924dbeb64be69480203fec/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.864186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9888492d5a448199f49e491de5fc87, user_id=50fe6c5b8b924dbeb64be69480203fec, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:31.867149 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b037ea-68b2-4698-a316-4bd404ea370d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:31.884359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2643] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:31 2022] PUT /identity/v3/projects/6c9888492d5a448199f49e491de5fc87/users/50fe6c5b8b924dbeb64be69480203fec/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:31.894003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.895570 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.895919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.897652 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956689d880dd482a9858df0fef255abb', 'request_id': 'req-edbd3a37-1982-49e2-adcc-7cd5fef85d35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.908503 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956689d880dd482a9858df0fef255abb', 'request_id': 'req-edbd3a37-1982-49e2-adcc-7cd5fef85d35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.932687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956689d880dd482a9858df0fef255abb', 'request_id': 'req-edbd3a37-1982-49e2-adcc-7cd5fef85d35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.939536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] MFA Rules not processed for user `50fe6c5b8b924dbeb64be69480203fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:31.941800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:31.949626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] The token cache is being invalidate because project 956689d880dd482a9858df0fef255abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:31.950065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.950791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-edbd3a37-1982-49e2-adcc-7cd5fef85d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:31.952650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2644] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:31 2022] DELETE /identity/v3/projects/956689d880dd482a9858df0fef255abb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:31.958545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883c7a-ad49-43bb-a8db-eaeb8aeccca6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:31.958886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883c7a-ad49-43bb-a8db-eaeb8aeccca6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:31.959219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39883c7a-ad49-43bb-a8db-eaeb8aeccca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:31.982405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39883c7a-ad49-43bb-a8db-eaeb8aeccca6 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:31.983767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39883c7a-ad49-43bb-a8db-eaeb8aeccca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.032807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2645] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:32.036551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:32.045360 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.047023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.061937 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2646] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:32.070227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.070511 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.070792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.092147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:32.093481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.105207 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2647] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:25:32.105690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:32.108021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.122579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e69933-f151-4d08-869f-c480f19f4df2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.124315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c06e4-7bdc-4c58-b014-740ab1c60149 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.126810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.127867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.128502 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.128925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.131687 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.133053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.133409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.133739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.134461 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:32.134840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.137196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.144116 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.146483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.156335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2648] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:32.160635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.168871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2649] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:32.171961 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c3ba-f84e-4b22-a1e3-3748267eed8e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.183481 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.184107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.184429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.184696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.185679 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.190111 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.227620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2650] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:32 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:32.233098 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.247291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac95682-015b-4cae-ab66-750293b2bb79 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.312872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.350483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.351490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.352248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.352897 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.353481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.356943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.424411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2651] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:32 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:32.426468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.456134 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5b1b-0162-404b-a6b4-729f380fe462 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.469334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51eb-b33a-4a1e-bb52-4c9d4e0acf87 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.469477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.470270 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.470414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.470681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.471329 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.472974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.483918 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2652] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:32.491357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.492235 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.492631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.492994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.497335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.501295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.515884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93853002-a461-4b0f-b661-bdc53ad81c09 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.528213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.528213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.528213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.528213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.529051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.530728 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.548306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2653] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:32.550937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.568589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c8fa-34fb-4fee-9de8-dfdb842528c7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.580305 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:32.583145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.583439 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.583724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] PATH_INFO: `/v3/projects/e7882d5c393f41fca3f7e313a95159e2/users/52899210af2f4acbb2d941668e8def22/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.616158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7882d5c393f41fca3f7e313a95159e2, user_id=52899210af2f4acbb2d941668e8def22, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.621422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502dd38-23fe-4d76-b821-3b870e4390d4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.625994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:32.627722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:32.649268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2654] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:32 2022] PUT /identity/v3/projects/e7882d5c393f41fca3f7e313a95159e2/users/52899210af2f4acbb2d941668e8def22/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:25:32.653772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2655] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:32.662151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:32.663058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:32.663393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:32.718902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] MFA Rules not processed for user `52899210af2f4acbb2d941668e8def22`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:32.721359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.836602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2656] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:32.852501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:32.854677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:32.955268 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe842c3-07c6-4255-af4a-e602306065ad service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:32.982909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:33.047946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:33.049106 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:33.049386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:33.049659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:33.060565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:33.062385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:33.074156 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2657] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:33.155965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:33.157429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:33.180915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:33.290190 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c0ec-324f-4c51-aaaf-c950aeb9c10a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:33.308911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:33.310939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:33.311229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:33.311597 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:33.315162 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:33.343311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:33.346749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:33.382878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2658] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:36.746384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:36.747926 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:36.862862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d803d-c0fe-47cf-99d5-8260e1d22f5e service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:36.890276 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:36.891034 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:36.891372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:36.891728 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] PATH_INFO: `/v3/users/0fb97bc956b14342990b1dd5d8c7be95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:36.893036 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb97bc956b14342990b1dd5d8c7be95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:36.895363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:36.993899 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb97bc956b14342990b1dd5d8c7be95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.002435 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb97bc956b14342990b1dd5d8c7be95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.012178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb97bc956b14342990b1dd5d8c7be95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.012964 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb97bc956b14342990b1dd5d8c7be95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.041562 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb97bc956b14342990b1dd5d8c7be95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.048080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2659] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:36 2022] DELETE /identity/v3/users/0fb97bc956b14342990b1dd5d8c7be95 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:37.366215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:37.380319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb2690-3f85-41bd-a0fd-d98e278c752c service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:37.466373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:37.467166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:37.467590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:37.468039 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] PATH_INFO: `/v3/projects/821c49e74c4e4767b872c300ac829345` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:37.469546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=821c49e74c4e4767b872c300ac829345)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:37.471901 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:37.543391 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '821c49e74c4e4767b872c300ac829345', 'request_id': 'req-99c89539-f6d4-4229-93cc-807b2457da00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.549186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '821c49e74c4e4767b872c300ac829345', 'request_id': 'req-99c89539-f6d4-4229-93cc-807b2457da00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.556254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '821c49e74c4e4767b872c300ac829345', 'request_id': 'req-99c89539-f6d4-4229-93cc-807b2457da00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.579130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] The token cache is being invalidate because project 821c49e74c4e4767b872c300ac829345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:37.579417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.580091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.581653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2660] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:37 2022] DELETE /identity/v3/projects/821c49e74c4e4767b872c300ac829345 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:37.586011 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:37.587496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:37.600994 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292fe8-0689-493e-82b4-522a00fbdbae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:37.640240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:37.662266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:37.662909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:37.663204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:37.663467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] PATH_INFO: `/v3/users/8a45508f4ce6417abac7c23d66fbd1e0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:37.664638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a45508f4ce6417abac7c23d66fbd1e0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:37.666632 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:37.718026 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a45508f4ce6417abac7c23d66fbd1e0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.724974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a45508f4ce6417abac7c23d66fbd1e0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.733725 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a45508f4ce6417abac7c23d66fbd1e0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.734331 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a45508f4ce6417abac7c23d66fbd1e0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.747702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a45508f4ce6417abac7c23d66fbd1e0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:37.753769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2661] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:37 2022] DELETE /identity/v3/users/8a45508f4ce6417abac7c23d66fbd1e0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:37.946168 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:37.947684 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:37.971876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c89539-f6d4-4229-93cc-807b2457da00 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:38.006986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:38.033242 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:38.033876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:38.034152 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:38.034423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] PATH_INFO: `/v3/projects/58c072b6f7cb4089a865ccc758faea65` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:38.035536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c072b6f7cb4089a865ccc758faea65)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:38.037545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:38.092390 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c072b6f7cb4089a865ccc758faea65', 'request_id': 'req-58343859-92a8-45ce-9be9-9a50e713ad41'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.098519 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c072b6f7cb4089a865ccc758faea65', 'request_id': 'req-58343859-92a8-45ce-9be9-9a50e713ad41'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.105679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c072b6f7cb4089a865ccc758faea65', 'request_id': 'req-58343859-92a8-45ce-9be9-9a50e713ad41'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.123985 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] The token cache is being invalidate because project 58c072b6f7cb4089a865ccc758faea65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:38.124542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.125212 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.126889 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2662] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:37 2022] DELETE /identity/v3/projects/58c072b6f7cb4089a865ccc758faea65 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:38.416850 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:38.418362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:38.432242 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07e4d1-e8b7-4da2-84d7-b8937fcae921 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:38.486793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:38.487424 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:38.487700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:38.487969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] PATH_INFO: `/v3/users/96590ab60e5e4583981915df7d0d0ba7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:38.489201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96590ab60e5e4583981915df7d0d0ba7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:38.491131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:38.544143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96590ab60e5e4583981915df7d0d0ba7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.551666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96590ab60e5e4583981915df7d0d0ba7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.555563 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:38.562300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96590ab60e5e4583981915df7d0d0ba7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.563088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96590ab60e5e4583981915df7d0d0ba7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.577854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96590ab60e5e4583981915df7d0d0ba7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.584162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2663] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:38 2022] DELETE /identity/v3/users/96590ab60e5e4583981915df7d0d0ba7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:38.770577 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:38.772733 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:38.794055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58343859-92a8-45ce-9be9-9a50e713ad41 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:38.860219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:38.860907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:38.861183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:38.861452 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] PATH_INFO: `/v3/projects/1d1df0bf099045dcb88142cbc4d31c07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:38.862620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1df0bf099045dcb88142cbc4d31c07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:38.864951 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:38.918825 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1df0bf099045dcb88142cbc4d31c07', 'request_id': 'req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.924818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1df0bf099045dcb88142cbc4d31c07', 'request_id': 'req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.931303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1df0bf099045dcb88142cbc4d31c07', 'request_id': 'req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.935085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:38.947097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] The token cache is being invalidate because project 1d1df0bf099045dcb88142cbc4d31c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:38.947379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.948019 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:38.949540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2664] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:38 2022] DELETE /identity/v3/projects/1d1df0bf099045dcb88142cbc4d31c07 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:38.953619 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:38.955137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:38.968506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee9bf0-5feb-446c-b1bc-9df986eed09a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.026966 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.027582 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.027854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.028102 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] PATH_INFO: `/v3/users/4ba1103d93c34a3ca2f8dffd16e30502` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.029272 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba1103d93c34a3ca2f8dffd16e30502)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.031201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.092988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba1103d93c34a3ca2f8dffd16e30502'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.102010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba1103d93c34a3ca2f8dffd16e30502'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.111772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba1103d93c34a3ca2f8dffd16e30502'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.112416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba1103d93c34a3ca2f8dffd16e30502'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.125720 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2405a3bb-a029-49ec-a8cf-237ab2506462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba1103d93c34a3ca2f8dffd16e30502'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.132183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2665] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:38 2022] DELETE /identity/v3/users/4ba1103d93c34a3ca2f8dffd16e30502 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:39.319128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.320899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:39.334986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e210c0-a9ca-4bd6-8357-abe70420cfc1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.399475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.400105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.400428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.400694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] PATH_INFO: `/v3/projects/7008ec18676444fa9e472406eaf9ac61` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.401824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7008ec18676444fa9e472406eaf9ac61)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.403726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.452997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7008ec18676444fa9e472406eaf9ac61', 'request_id': 'req-2a10546b-d408-4ac6-a934-5b058087b25c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.458673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7008ec18676444fa9e472406eaf9ac61', 'request_id': 'req-2a10546b-d408-4ac6-a934-5b058087b25c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.464747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7008ec18676444fa9e472406eaf9ac61', 'request_id': 'req-2a10546b-d408-4ac6-a934-5b058087b25c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.479121 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] The token cache is being invalidate because project 7008ec18676444fa9e472406eaf9ac61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:39.479399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.480097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2a10546b-d408-4ac6-a934-5b058087b25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:39.481618 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2666] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:39 2022] DELETE /identity/v3/projects/7008ec18676444fa9e472406eaf9ac61 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:39.486855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.487129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.487406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.506971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:39.508161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:39.580677 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:39.593796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2667] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:39.602449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.602724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.602964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.625887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:39.627082 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:39.636145 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2668] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:39.640548 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.651104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae93ab4-991e-4fde-857a-6bc3743422fa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.656862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.657463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.657736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.658014 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.658643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:39.658942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.660576 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.672722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2669] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:39.677792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.686940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a861d-71b8-42ad-b6f5-1b9b5f635ec5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.691709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.692865 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.693142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.693411 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.694051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.695921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.724828 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2670] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:39 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:39.729222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.740686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566410d8-1cf9-4803-b29f-c7c31954741f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.800600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.801217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.801575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.802005 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.802439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.804372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.826822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2671] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:39 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:39.831172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.841938 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832c06-2f6d-4f84-9858-6a2de267395d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.848069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.849328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.849601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.849866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.850537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.852491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.863132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2672] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:39.867846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.878835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dc081-2259-40d0-a8c3-bc99e4dd1e9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.884505 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.885107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.885381 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.885647 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.886282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.887873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89591d7b-6a11-486c-8458-85542c67f470 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.900798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2673] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:39.906998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:39.918492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1b13-ed68-45ff-9879-46ade58bb36c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:39.926068 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:39.927188 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.927332 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.927505 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] PATH_INFO: `/v3/projects/6e6ef746eca046709573764a92428fb5/users/952eca48a1934e93b8f4f220d26e988e/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:39.944015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6ef746eca046709573764a92428fb5, user_id=952eca48a1934e93b8f4f220d26e988e, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:39.946717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:39.962453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2674] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:39 2022] PUT /identity/v3/projects/6e6ef746eca046709573764a92428fb5/users/952eca48a1934e93b8f4f220d26e988e/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:39.974100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:39.974100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:39.974100 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.003478 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] MFA Rules not processed for user `952eca48a1934e93b8f4f220d26e988e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:40.004898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:40.073640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2675] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:40.079049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.108493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:40.149560 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f076de-d489-4d22-a0ac-e25336b7943e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.155327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.155928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.156197 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.156504 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.157265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.159009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.197294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2676] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:40 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:40.198773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.210644 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7e9f9-3c57-4d75-af38-9ee85ea8be5b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.266226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:40.278950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.279579 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.279856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.280124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.280788 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.282910 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.309710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2677] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:40 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:40.313960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.325325 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cac5b-f1bd-48cd-8745-7c850bf0b1b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.333869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.334537 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.334817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.335084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.335702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.337366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.349605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2678] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:40.353854 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.366858 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49245329-3042-46b3-8a29-687605f74e81 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.371886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.372591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.372862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.373127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] PATH_INFO: `/v3/projects/7cf15852b4844afa96695a36ccd9abc8/users/aac79e15bafa4391a65ea41546a476f8/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.395664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf15852b4844afa96695a36ccd9abc8, user_id=aac79e15bafa4391a65ea41546a476f8, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.399266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.415067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2679] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:40 2022] PUT /identity/v3/projects/7cf15852b4844afa96695a36ccd9abc8/users/aac79e15bafa4391a65ea41546a476f8/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:40.419006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.430453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da5cc-959e-48a8-a19e-404152303515 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.499278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.500035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.500402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.500762 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.501602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.502994 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f3f87-1d8e-42f1-b477-cb1a7dedd15d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.513831 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2680] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:40.518213 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.532584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18688329-8907-47f4-8d6f-0b5de390d48a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.539323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.540169 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.540611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.540881 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] PATH_INFO: `/v3/projects/7cf15852b4844afa96695a36ccd9abc8/users/aac79e15bafa4391a65ea41546a476f8/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.542922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf15852b4844afa96695a36ccd9abc8, user_id=aac79e15bafa4391a65ea41546a476f8, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.545628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.558227 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2681] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:40 2022] PUT /identity/v3/projects/7cf15852b4844afa96695a36ccd9abc8/users/aac79e15bafa4391a65ea41546a476f8/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:40.569603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.569885 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.570153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.600602 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] MFA Rules not processed for user `aac79e15bafa4391a65ea41546a476f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:40.601808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:40.702685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2682] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:40.744098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:40.744098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:40.812875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d471-d828-46a0-8dca-807f76bccbc6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:40.817396 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:40.818264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:40.818601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:40.818939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:40.880151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:40.881749 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:40.891561 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2683] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:25:45.325269 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:45.326920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:45.342071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e829ea-7d7e-46de-a7b7-b3cbbdcdbcc8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:45.346254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:45.346886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:45.347158 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:45.347420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:45.359711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:45.361228 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c79c8-ec41-4ac4-aeee-822cedc57653 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:45.372740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2684] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:50.364208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:50.447601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:50.499394 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6691fe3-c39d-4daf-9b10-d937ad73fd09 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:50.504627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:50.505297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:50.505567 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:50.505829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] PATH_INFO: `/v3/users/952eca48a1934e93b8f4f220d26e988e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:50.506958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=952eca48a1934e93b8f4f220d26e988e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:50.509177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:50.560532 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952eca48a1934e93b8f4f220d26e988e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:50.567618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952eca48a1934e93b8f4f220d26e988e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:50.576079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952eca48a1934e93b8f4f220d26e988e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:50.578119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952eca48a1934e93b8f4f220d26e988e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:50.590740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952eca48a1934e93b8f4f220d26e988e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:50.597463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2685] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:50 2022] DELETE /identity/v3/users/952eca48a1934e93b8f4f220d26e988e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:50.602161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:50.602650 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:50.602771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:50.609592 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:50.655949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:50.657154 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:50.721401 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2686] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:50.731543 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:50.733754 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:50.815180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611825c-fb60-4963-a2a9-9f51bd5839fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:50.818909 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:50.819521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:50.819843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:50.820226 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:50.832334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:50.833835 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:50.844248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2687] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:51.134275 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:51.136031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:51.153647 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794013f-2118-4ddc-97f3-7e266bab2097 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:51.160869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:51.161545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:51.161932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:51.162210 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] PATH_INFO: `/v3/projects/6e6ef746eca046709573764a92428fb5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:51.163713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6ef746eca046709573764a92428fb5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:51.165946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:51.217290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6ef746eca046709573764a92428fb5', 'request_id': 'req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.222664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6ef746eca046709573764a92428fb5', 'request_id': 'req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.229634 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6ef746eca046709573764a92428fb5', 'request_id': 'req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.247607 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] The token cache is being invalidate because project 6e6ef746eca046709573764a92428fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:51.247895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.248575 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.250063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2688] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:51 2022] DELETE /identity/v3/projects/6e6ef746eca046709573764a92428fb5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:51.254340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:51.264417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b072-6aba-42e1-8744-fa8061eef056 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:51.286865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:51.328314 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:51.328976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:51.329247 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:51.329510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] PATH_INFO: `/v3/users/aac79e15bafa4391a65ea41546a476f8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:51.330711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac79e15bafa4391a65ea41546a476f8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:51.332699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:51.385634 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac79e15bafa4391a65ea41546a476f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.393794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac79e15bafa4391a65ea41546a476f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.405413 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac79e15bafa4391a65ea41546a476f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.406090 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac79e15bafa4391a65ea41546a476f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.419416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96428235-b4e5-4776-bd1f-3b544999d6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac79e15bafa4391a65ea41546a476f8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.426140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2689] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:51 2022] DELETE /identity/v3/users/aac79e15bafa4391a65ea41546a476f8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:51.730493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:51.730493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:51.750693 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6585f5-d0c1-4401-a5b3-5587e05d6adb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:51.811000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:51.811737 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:51.812104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:51.812104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] PATH_INFO: `/v3/projects/7cf15852b4844afa96695a36ccd9abc8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:51.813410 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf15852b4844afa96695a36ccd9abc8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:51.815430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:51.818835 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:51.872206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf15852b4844afa96695a36ccd9abc8', 'request_id': 'req-889c17d0-2ccb-4022-9c0a-1686406ef32a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.884000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf15852b4844afa96695a36ccd9abc8', 'request_id': 'req-889c17d0-2ccb-4022-9c0a-1686406ef32a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.893177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf15852b4844afa96695a36ccd9abc8', 'request_id': 'req-889c17d0-2ccb-4022-9c0a-1686406ef32a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.908110 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] The token cache is being invalidate because project 7cf15852b4844afa96695a36ccd9abc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:51.908419 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.909165 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-889c17d0-2ccb-4022-9c0a-1686406ef32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:51.910947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2690] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:51 2022] DELETE /identity/v3/projects/7cf15852b4844afa96695a36ccd9abc8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:51.917354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:51.917354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:51.917626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:51.972602 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:51.973700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:52.036170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2691] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:52.105309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.105585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.105874 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.125407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:52.126584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:52.134944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2692] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:52.140113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.151517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd50ea-1e57-4bf5-bf2f-f561299c7b0d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.158165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.158802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.159104 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.159373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.160033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:52.160361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.162009 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.177487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2693] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:52.182309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.192092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60847a8c-49a4-49e7-b31c-336cb65ae782 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.198208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.198837 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.199107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.199428 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.200044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.202052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.237328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2694] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:52 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:52.241837 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.253006 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5631d-24ff-49d2-89e4-fd99af3e14d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.257049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:52.317666 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.318278 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.318550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.318812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.319442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.321391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.347583 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2695] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:52 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:25:52.353836 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.366513 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d712c5-bdca-4f77-ab84-212d30450604 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.375949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.376527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.376866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.377047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.377702 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.379321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.391069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2696] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:52.395362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.406570 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c600b-32be-4af3-b046-e37476d6f51a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.411771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.412433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.412727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.412981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.413623 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.415257 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc8a4-c95a-4c83-a733-799455921490 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.426200 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2697] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:52.432670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.444538 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3068506-72e7-4048-81f3-f65290be41a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.451271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.452123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.452430 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.452710 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] PATH_INFO: `/v3/projects/8398bb686250432db038fdb06fe7cc01/users/ba4099ea415b4594b710219069219034/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.469229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=8398bb686250432db038fdb06fe7cc01, user_id=ba4099ea415b4594b710219069219034, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.471853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.487931 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2698] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:52 2022] PUT /identity/v3/projects/8398bb686250432db038fdb06fe7cc01/users/ba4099ea415b4594b710219069219034/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:52.499360 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.499640 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.499906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.531271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] MFA Rules not processed for user `ba4099ea415b4594b710219069219034`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:52.533128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:52.612488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2699] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:52.645520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.647331 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:52.705719 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b8f5-771e-498c-9e7f-e6a26fbf5cde admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.709450 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.710053 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.710328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.710596 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.719457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.720930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63fdc-4a3c-4440-ac79-550063205411 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.730184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2700] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:52.748802 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:52.819265 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71225aff-e87e-43a4-bf2a-223f7f9d1682 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:52.824204 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:52.824875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.825161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.825433 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] PATH_INFO: `/v3/users/ba4099ea415b4594b710219069219034` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.826723 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4099ea415b4594b710219069219034)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:52.828722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:52.845772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:52.881350 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4099ea415b4594b710219069219034'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:52.893262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4099ea415b4594b710219069219034'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:52.902920 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4099ea415b4594b710219069219034'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:52.903726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4099ea415b4594b710219069219034'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:52.917386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4099ea415b4594b710219069219034'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:52.924827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2701] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:25:52 2022] DELETE /identity/v3/users/ba4099ea415b4594b710219069219034 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:52.929215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:52.929730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:52.929730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:52.952481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:52.953711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:52.980749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:53.030274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2702] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:53.039055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.040631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:53.104559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce225c-dc55-4e6c-ba04-12d33e4646f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.109510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.110168 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.110405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.110708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.121177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.122803 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767e62-7076-47a9-acd5-eeb61bbe5d09 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.136064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2703] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:25:53.418253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.420014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:53.433048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec4e7-50b3-4136-9f0e-0d901e06d677 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.438879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.439678 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.439997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.440284 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] PATH_INFO: `/v3/projects/8398bb686250432db038fdb06fe7cc01` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.441551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8398bb686250432db038fdb06fe7cc01)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.443527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.500944 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8398bb686250432db038fdb06fe7cc01', 'request_id': 'req-fe8c5ba2-5410-4627-b443-57b184c0ea29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:53.506490 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8398bb686250432db038fdb06fe7cc01', 'request_id': 'req-fe8c5ba2-5410-4627-b443-57b184c0ea29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:53.512738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8398bb686250432db038fdb06fe7cc01', 'request_id': 'req-fe8c5ba2-5410-4627-b443-57b184c0ea29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:53.528999 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] The token cache is being invalidate because project 8398bb686250432db038fdb06fe7cc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:25:53.529547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:53.530303 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe8c5ba2-5410-4627-b443-57b184c0ea29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:25:53.531892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2704] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:25:53 2022] DELETE /identity/v3/projects/8398bb686250432db038fdb06fe7cc01 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:53.537372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.537834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.537834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.560361 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:53.561627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:53.624495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2705] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:53.633485 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.633850 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.634189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.656732 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:53.657923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:53.666215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2706] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:53.671369 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.685843 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220be3a-3abc-4552-9aa8-8500614dc684 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.693327 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.693959 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.694229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.694493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.695155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:25:53.695456 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.697151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.710611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2707] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:25:53.715221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.725385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd3125-f685-4183-84ba-86d1f0567aae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.730913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.732037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.732416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.732691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.733393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.735356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.767534 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2708] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:25:53 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:53.771897 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.782694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784e642-eb2b-4dbc-93f4-f528149ae68d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.805408 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:53.844972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.845572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.845862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.846126 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.846735 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.848707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.874988 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2709] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:25:53 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:25:53.879320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.890576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ae1c-5b15-4a4f-9fd4-04fd5312fc17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.896500 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.897105 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.897385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.897649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.898682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.900259 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.911097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2710] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:25:53.915377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.927229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580131da-b126-4d43-a6f7-2681cce4f1c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.932130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.932860 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.933178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.933486 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:53.934243 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:53.936101 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb8402-2c26-48d4-bf6c-305b261100c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:53.952927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2711] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:25:53.957071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:53.970061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1ea5-3b26-4eac-b5b1-bc43b603dfeb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:53.974778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:53.975506 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:53.975751 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:53.976186 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] PATH_INFO: `/v3/projects/3721087bff1243db80b5d24ccc0ee228/users/611811104b394982bee09de20844f64b/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:54.007179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3721087bff1243db80b5d24ccc0ee228, user_id=611811104b394982bee09de20844f64b, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:54.007179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:54.022501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2712] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:25:53 2022] PUT /identity/v3/projects/3721087bff1243db80b5d24ccc0ee228/users/611811104b394982bee09de20844f64b/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:25:54.034729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2958de-973a-4c4e-91d7-119309bbbefc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:54.035010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2958de-973a-4c4e-91d7-119309bbbefc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:54.035277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2958de-973a-4c4e-91d7-119309bbbefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:54.064121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d2958de-973a-4c4e-91d7-119309bbbefc None None] MFA Rules not processed for user `611811104b394982bee09de20844f64b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:25:54.065281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d2958de-973a-4c4e-91d7-119309bbbefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:54.135283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2713] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:25:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:25:54.147572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:25:54.149808 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:25:54.215659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bfe6a-aef2-4a45-9f64-8d0c8663616d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:25:54.219866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:25:54.220520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:25:54.220794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:25:54.221061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:25:54.226515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:25:54.230670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:25:54.232852 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:25:54.243044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2714] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:25:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:04.132153 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:04.239933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:10.286142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff19317-da60-4608-a2ab-df500877a96f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:10.286731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff19317-da60-4608-a2ab-df500877a96f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:10.286731 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff19317-da60-4608-a2ab-df500877a96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:10.307727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eff19317-da60-4608-a2ab-df500877a96f None None] MFA Rules not processed for user `5b4c4030b5fb4381a598a54588880b75`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:10.309150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eff19317-da60-4608-a2ab-df500877a96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:10.370094 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2715] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:20.363542 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:27.206670 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:27.271662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524743ec-6bf2-48ef-a6bd-6f1d39ca3fb3 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:27.276672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:27.277300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:27.277578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:27.277848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] PATH_INFO: `/v3/users/611811104b394982bee09de20844f64b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:27.278997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=611811104b394982bee09de20844f64b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:27.280946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:27.331725 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611811104b394982bee09de20844f64b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:27.332415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611811104b394982bee09de20844f64b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:27.344253 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611811104b394982bee09de20844f64b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:27.357064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611811104b394982bee09de20844f64b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:27.357741 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611811104b394982bee09de20844f64b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:27.370463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2716] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:27 2022] DELETE /identity/v3/users/611811104b394982bee09de20844f64b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:27.375411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:27.375927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:27.376995 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:27.397250 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:27.398444 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:27.472495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2717] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:27.480521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:27.481997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:27.574962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710e381-b920-4485-9040-cddcfb6c8aa6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:27.579128 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:27.579742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:27.580017 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:27.580384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:27.590579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:27.592038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:27.601717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2718] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:27.940193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:27.941838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:27.957218 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab9bcb-5b5f-4fa5-adbb-8e22ecc0b446 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:27.962958 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:27.963145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:27.963316 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:27.963715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] PATH_INFO: `/v3/projects/3721087bff1243db80b5d24ccc0ee228` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:27.964887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3721087bff1243db80b5d24ccc0ee228)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:27.966787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:28.021239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3721087bff1243db80b5d24ccc0ee228', 'request_id': 'req-50657155-e7b7-4392-8e27-90bca3f2b90e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:28.027620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3721087bff1243db80b5d24ccc0ee228', 'request_id': 'req-50657155-e7b7-4392-8e27-90bca3f2b90e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:28.038467 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3721087bff1243db80b5d24ccc0ee228', 'request_id': 'req-50657155-e7b7-4392-8e27-90bca3f2b90e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:28.050856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] The token cache is being invalidate because project 3721087bff1243db80b5d24ccc0ee228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:28.051162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:28.051825 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-50657155-e7b7-4392-8e27-90bca3f2b90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:28.053374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2719] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:27 2022] DELETE /identity/v3/projects/3721087bff1243db80b5d24ccc0ee228 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:30.739306 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:30.740921 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:30.749418 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:30.756001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32cbff-8d97-495d-a8ed-0013067f24e3 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:30.818501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:30.819130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:30.819477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:30.819653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] PATH_INFO: `/v3/users/1b248640a229456e872aaab669843ec9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:30.821074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b248640a229456e872aaab669843ec9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:30.823094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:30.873254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b248640a229456e872aaab669843ec9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:30.873913 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b248640a229456e872aaab669843ec9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:30.882720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b248640a229456e872aaab669843ec9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:30.896338 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b248640a229456e872aaab669843ec9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:30.897209 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b248640a229456e872aaab669843ec9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:30.911142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2720] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:30 2022] DELETE /identity/v3/users/1b248640a229456e872aaab669843ec9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:30.916962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:30.916962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:30.916962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:30.936145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:30.937407 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.000609 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2721] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:31.009916 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.011402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.019368 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.020785 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.033275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea0395-672f-4ed3-8249-42952c29fe2b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.038643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.039580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.039580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.039805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] PATH_INFO: `/v3/users/0d78d75c8082439d9d99fccbf9da61e1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.040874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d78d75c8082439d9d99fccbf9da61e1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.042971 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.065748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:31.071290 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51544ae5-0c19-44c3-9b86-3c151a9113bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.076207 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.076816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.077091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.077384 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.088026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.089544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418ee2e-a137-4946-a3d7-75071b1dc236 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.099478 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2722] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:31.102451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d78d75c8082439d9d99fccbf9da61e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.110441 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d78d75c8082439d9d99fccbf9da61e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.120477 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d78d75c8082439d9d99fccbf9da61e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.121374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d78d75c8082439d9d99fccbf9da61e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.134157 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d78d75c8082439d9d99fccbf9da61e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.141178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2723] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:31 2022] DELETE /identity/v3/users/0d78d75c8082439d9d99fccbf9da61e1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:31.145757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.146042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.146315 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.166141 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:31.167340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.186952 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:31.231463 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2724] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:31.241700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.243378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.305707 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715df1af-7bf0-48ea-b178-7a2998e7bea1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.312746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.312746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.312746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.312746 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.321079 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.322541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8a253-2b4f-4e7e-9ab4-c1e728edfcda service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.334149 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2725] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:31.563057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.567783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.588683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf895c-1dc7-4a5e-aa4d-b21632d4d664 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.601247 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.601826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.602097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.602375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] PATH_INFO: `/v3/projects/0cc911d8ab344a06a7c8f6034bbdfbcb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.603700 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc911d8ab344a06a7c8f6034bbdfbcb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.606164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.666991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc911d8ab344a06a7c8f6034bbdfbcb', 'request_id': 'req-e4871bcc-cac2-492d-8a14-895acb402502'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.674386 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc911d8ab344a06a7c8f6034bbdfbcb', 'request_id': 'req-e4871bcc-cac2-492d-8a14-895acb402502'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.684349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc911d8ab344a06a7c8f6034bbdfbcb', 'request_id': 'req-e4871bcc-cac2-492d-8a14-895acb402502'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.705727 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] The token cache is being invalidate because project 0cc911d8ab344a06a7c8f6034bbdfbcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:31.706576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.707224 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.709834 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2726] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:31 2022] DELETE /identity/v3/projects/0cc911d8ab344a06a7c8f6034bbdfbcb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:31.716214 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b320-9297-4ce7-a2d7-f7e681fc0d95 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.716662 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b320-9297-4ce7-a2d7-f7e681fc0d95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.717193 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58b320-9297-4ce7-a2d7-f7e681fc0d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.733064 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.736048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.760539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f58b320-9297-4ce7-a2d7-f7e681fc0d95 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:31.762310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f58b320-9297-4ce7-a2d7-f7e681fc0d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.818459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4871bcc-cac2-492d-8a14-895acb402502 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.823990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.824785 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.825166 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.825533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] PATH_INFO: `/v3/projects/c3cc2647f5a743d381a29e6374edc23d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.826798 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3cc2647f5a743d381a29e6374edc23d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.828842 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.842842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2727] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:31.851277 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.851612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.851895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.874344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:31.875618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.888203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3cc2647f5a743d381a29e6374edc23d', 'request_id': 'req-5922ce81-bf01-4734-ae12-f30df22d3290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.890144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:31.897460 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3cc2647f5a743d381a29e6374edc23d', 'request_id': 'req-5922ce81-bf01-4734-ae12-f30df22d3290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.910278 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3cc2647f5a743d381a29e6374edc23d', 'request_id': 'req-5922ce81-bf01-4734-ae12-f30df22d3290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.919963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:31.934496 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] The token cache is being invalidate because project c3cc2647f5a743d381a29e6374edc23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:31.934643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.935295 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:31.937100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2728] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:31 2022] DELETE /identity/v3/projects/c3cc2647f5a743d381a29e6374edc23d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:31.941799 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.943357 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:31.957065 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2729] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:31.960962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:31.970556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d53ed-557e-49a5-a3b3-133169ce9e88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:31.975442 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:31.976093 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:31.976391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:31.976657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:31.977552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:31.977691 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:31.979299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:31.993353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2730] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:26:31.997625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.009380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17597d0e-7732-4986-a484-43fe60e82a0d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.015498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.016130 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.016430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.016695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.017319 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.019211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.025058 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922ce81-bf01-4734-ae12-f30df22d3290 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.029934 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.030797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.030927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.031100 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] PATH_INFO: `/v3/users/50fe6c5b8b924dbeb64be69480203fec` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.032248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=50fe6c5b8b924dbeb64be69480203fec)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.034181 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.052509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2731] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:31 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:32.054057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.087025 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50fe6c5b8b924dbeb64be69480203fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.087770 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50fe6c5b8b924dbeb64be69480203fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.097568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50fe6c5b8b924dbeb64be69480203fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.109892 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50fe6c5b8b924dbeb64be69480203fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.110603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50fe6c5b8b924dbeb64be69480203fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.122712 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2732] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:31 2022] DELETE /identity/v3/users/50fe6c5b8b924dbeb64be69480203fec => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:32.123445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972c970-36db-45a7-a976-5c16ee6f5c44 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.190139 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.190739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.191027 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.191451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.192152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.194174 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.221688 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2733] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:32.222887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.224366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.238208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fc22-cfd9-480c-8bf1-8a2bdf7f4bcb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.243208 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.243920 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.244434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.244772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.245528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.247203 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.261810 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2734] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:32.263051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.274095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e38-4d37-4c96-8253-7466d34d19e5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.279598 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.280351 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.280694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.281018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.282064 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.284004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3b2e9-2543-422a-8ae8-f75bc8419207 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.321416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2735] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:32.321919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.335050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67664-2054-4554-b0c6-de0701215fab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.340910 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.341672 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.342015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.342355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] PATH_INFO: `/v3/projects/2b3b0a56e9724ccd81df765dfc49739d/users/909eaedf75674f88b41cbdc0eac33964/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.370230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3b0a56e9724ccd81df765dfc49739d, user_id=909eaedf75674f88b41cbdc0eac33964, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.372928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.393090 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2736] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:32 2022] PUT /identity/v3/projects/2b3b0a56e9724ccd81df765dfc49739d/users/909eaedf75674f88b41cbdc0eac33964/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:32.398203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.398485 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.399143 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.421281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:32.423140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.453418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:32.490134 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2737] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:32.505049 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.506871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.516875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.522524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.532702 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e685b-f51a-4075-867e-bd6a33e4d7c5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.540546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.541698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.542041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.542317 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] PATH_INFO: `/v3/projects/6c9888492d5a448199f49e491de5fc87` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.543752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9888492d5a448199f49e491de5fc87)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.545327 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:32.545965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.580797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7980e9-f0a5-4762-b107-8130f6743b27 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.584997 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.585610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.585895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.586151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.595556 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.596029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9888492d5a448199f49e491de5fc87', 'request_id': 'req-8bde0d58-59a3-465a-92d0-7d09584ef5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.597232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a0595-94c5-495a-a180-d5edc60a8c7b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.601742 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9888492d5a448199f49e491de5fc87', 'request_id': 'req-8bde0d58-59a3-465a-92d0-7d09584ef5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.607812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2738] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:32.611349 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9888492d5a448199f49e491de5fc87', 'request_id': 'req-8bde0d58-59a3-465a-92d0-7d09584ef5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.623118 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] The token cache is being invalidate because project 6c9888492d5a448199f49e491de5fc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:32.623639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.628804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bde0d58-59a3-465a-92d0-7d09584ef5a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:32.630713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2739] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:32 2022] DELETE /identity/v3/projects/6c9888492d5a448199f49e491de5fc87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:32.635966 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.636238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.636578 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.656527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:32.657797 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.722624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2740] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:32.732359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.732636 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.732856 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.753772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:32.755088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:32.763900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2741] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:32.768544 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.782631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c17f3f-65c1-4b39-9f5b-1469799aa9bc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.788487 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.789288 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.789660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.790073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.791034 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:32.791657 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.793414 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.808245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2742] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:26:32.813389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.824626 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecdba9-a8c8-48ec-a820-c8e92c5d7d5d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.831192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.832096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.832495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.832869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.833873 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.835867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:32.874190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2743] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/projects => generated 361 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:32.878827 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:32.890520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db36fe-3a44-4eb6-9b28-31ceabec4c08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:32.978571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:32.979280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:32.979717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:32.980211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:32.981397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:32.983528 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.009115 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2744] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:32 2022] POST /identity/v3/users => generated 281 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:26:33.014216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.028924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb68b16-744c-487f-be92-6d621b840acf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.033922 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.034580 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.034917 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.035273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.035986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.037748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.050621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2745] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:33.054656 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.065734 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827514-9cfc-4b31-8b46-0f329ee6f3a1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.071245 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.071960 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.072674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.073198 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.073946 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.075611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1f2ca-8bbe-47fb-b48c-dc74963cd356 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.087160 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2746] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:33.091842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.102256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566f578-2391-48ff-9d74-aea15c019f05 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.107399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.108110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.108489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.108822 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] PATH_INFO: `/v3/projects/14684a16eed9487483230deca11dd99b/users/23c42be9e163453f871c1c7af05e52fa/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.124033 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14684a16eed9487483230deca11dd99b, user_id=23c42be9e163453f871c1c7af05e52fa, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.126974 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.141463 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2747] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:33 2022] PUT /identity/v3/projects/14684a16eed9487483230deca11dd99b/users/23c42be9e163453f871c1c7af05e52fa/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:33.153416 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.153724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.153969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.186472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] MFA Rules not processed for user `23c42be9e163453f871c1c7af05e52fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:33.188753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:33.211864 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:33.289182 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2748] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:33.295196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.368701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:33.386071 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811636-93d3-4477-a27b-bc2b1e14b585 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.392986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.394804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.395125 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.395700 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.396648 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.399359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.428903 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2749] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:33 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:33.433125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.450180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac02812-8e50-4ca6-a27c-c6f5b5b2c8ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.509854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.510845 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.511183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.511511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.512215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.514271 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.548880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2750] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:33 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:26:33.552631 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.565983 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82bb9e-8ea2-49ae-b379-d63f22cbd009 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.573203 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.574798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.575152 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.575488 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.576299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.578216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.592705 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2751] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:33.596912 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.608192 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ac682-c28c-4739-9850-5a3683bdb18e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.613633 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.614422 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.614759 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.615093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] PATH_INFO: `/v3/projects/9c4356340add4d1abc552d3798fc52c4/users/eda751290dc34329933eea96cd08632d/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.632749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4356340add4d1abc552d3798fc52c4, user_id=eda751290dc34329933eea96cd08632d, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.635789 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.652620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2752] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:33 2022] PUT /identity/v3/projects/9c4356340add4d1abc552d3798fc52c4/users/eda751290dc34329933eea96cd08632d/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:33.656812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.667307 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355473-d18a-43ab-b0cf-34706651f8af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.735885 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.736534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.736813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.737056 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.737861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.739529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87035099-b480-404d-ad54-e362b6ed80c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.750494 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2753] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:33.755211 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:33.768523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64182a7-6892-4969-952c-dcceac2df4c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:33.773047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:33.774061 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.774333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.774606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] PATH_INFO: `/v3/projects/9c4356340add4d1abc552d3798fc52c4/users/eda751290dc34329933eea96cd08632d/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.777205 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4356340add4d1abc552d3798fc52c4, user_id=eda751290dc34329933eea96cd08632d, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:33.780172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:33.792470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2754] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:33 2022] PUT /identity/v3/projects/9c4356340add4d1abc552d3798fc52c4/users/eda751290dc34329933eea96cd08632d/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:33.805367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:33.806344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:33.807305 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:33.861597 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] MFA Rules not processed for user `eda751290dc34329933eea96cd08632d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:33.863740 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:34.015385 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2755] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:34.023491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:34.026314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:34.189260 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1962e-d987-4120-be41-7663be5f91f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:34.194420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:34.195536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:34.195878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:34.196289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:34.213914 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:34.217433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:34.221856 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:34.241075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2756] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:34.345379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:34.359041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb41c1-d3ae-45ba-a883-379bc2604c02 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:34.364192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:34.365201 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:34.365531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:34.365863 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:34.421037 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:34.438697 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:34.440401 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0bd66-21d7-4acc-be1f-52b21bd2c3e6 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:34.450446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:35.266919 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:35.268646 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:35.343630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bf150-3922-4c92-8dd6-7b69ba452ec0 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:35.348511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:35.349150 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:35.349477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:35.349959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] PATH_INFO: `/v3/users/52899210af2f4acbb2d941668e8def22` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:35.351178 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=52899210af2f4acbb2d941668e8def22)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:35.353571 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:35.449866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52899210af2f4acbb2d941668e8def22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:35.450637 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52899210af2f4acbb2d941668e8def22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:35.461333 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52899210af2f4acbb2d941668e8def22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:35.479612 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52899210af2f4acbb2d941668e8def22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:35.480896 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52899210af2f4acbb2d941668e8def22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:35.502935 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2758] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:35 2022] DELETE /identity/v3/users/52899210af2f4acbb2d941668e8def22 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:35.507129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:35.507557 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:35.508927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:35.547892 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:35.549706 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:35.633821 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2759] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:35.647292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:35.649475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:35.777431 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e67e4c-d91e-408a-902f-b286a4f5daaa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:35.781665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:35.782992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:35.783429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:35.783857 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:35.796628 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:35.798407 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ab440-5e9d-47dd-81e9-f5f13a4cf01c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:35.813415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2760] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:36.208297 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:36.209811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:36.229506 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919cc25-5642-4e70-b964-1a43cb36e055 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:36.241053 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:36.241710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.241980 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.242311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] PATH_INFO: `/v3/projects/e7882d5c393f41fca3f7e313a95159e2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.253217 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7882d5c393f41fca3f7e313a95159e2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:36.256304 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:36.327823 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7882d5c393f41fca3f7e313a95159e2', 'request_id': 'req-dab34a28-96c0-4825-a42a-ec32b535ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:36.335018 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7882d5c393f41fca3f7e313a95159e2', 'request_id': 'req-dab34a28-96c0-4825-a42a-ec32b535ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:36.355384 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7882d5c393f41fca3f7e313a95159e2', 'request_id': 'req-dab34a28-96c0-4825-a42a-ec32b535ab73'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:36.400787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] The token cache is being invalidate because project e7882d5c393f41fca3f7e313a95159e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:36.406280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:36.409722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dab34a28-96c0-4825-a42a-ec32b535ab73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:36.422904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2761] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:36 2022] DELETE /identity/v3/projects/e7882d5c393f41fca3f7e313a95159e2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:36.431132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.431742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.431742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.511735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:36.513839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:36.556262 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:36.613893 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2762] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:36.624260 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.624670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.624982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.651573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:36.652960 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:36.664510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2763] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:36.665558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:36.683980 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7389275-86de-4dad-96fb-e233f1ae2abb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:36.697582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:36.699685 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.700484 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.701436 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.703388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:36.703803 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:36.708526 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:36.726521 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2764] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:26:36.728060 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:36.741595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338a7e4-cdfd-4c56-ba96-678351ba464e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:36.746544 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:36.747274 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.747547 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.747812 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.748468 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:36.750668 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:36.841180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2765] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:36 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:36.845161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:36.862003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e298-b8aa-4f14-bfe5-87a7ce868b3f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:36.968692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:36.969474 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:36.969805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:36.970517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:36.971023 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:36.973054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.031735 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2766] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:36 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:37.033097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.043969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e4c0-30b4-4076-ac7e-5e7c3f5e8ae3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:37.055096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:37.055734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.056008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.056296 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.057440 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:37.059080 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.076318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2767] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:37.080318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.091060 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb7804-acda-4a9e-8287-d984ea6173d5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:37.095812 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:37.096461 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.096738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.097013 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.097659 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:37.099256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190f07-4f82-4b87-914f-3cd4dad5206f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.111873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2768] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:37.115445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.129830 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e37e4-9d46-4618-9da4-755c35670cc0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:37.135560 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:37.136379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.136655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.137479 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] PATH_INFO: `/v3/projects/1f51871afe1c49f4a472b09e633ad1b9/users/34efeb60abd8417ba53962a45fd3e5a0/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.153086 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f51871afe1c49f4a472b09e633ad1b9, user_id=34efeb60abd8417ba53962a45fd3e5a0, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:37.155756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.171326 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2769] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:37 2022] PUT /identity/v3/projects/1f51871afe1c49f4a472b09e633ad1b9/users/34efeb60abd8417ba53962a45fd3e5a0/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:37.183663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.184045 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.184423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.214778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] MFA Rules not processed for user `34efeb60abd8417ba53962a45fd3e5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:37.216089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:37.313050 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2770] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:37.326828 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.328883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:37.335923 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:37.423838 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd9d67-e1f7-48b8-b4d6-1e244903b45e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:37.428259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:37.429039 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.429383 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.429719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.448052 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:37.450429 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2a6d1-931d-4a16-9f40-f0531b4a71a3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.464202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:37.553729 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.556044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:37.575545 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:37.680006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9717261-c120-4b63-a84f-12263f7c07da None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:37.685769 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:37.689143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.689677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.690226 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] PATH_INFO: `/v3/users/23c42be9e163453f871c1c7af05e52fa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.691792 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c42be9e163453f871c1c7af05e52fa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:37.694146 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:37.782254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c42be9e163453f871c1c7af05e52fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:37.783340 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c42be9e163453f871c1c7af05e52fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:37.799880 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c42be9e163453f871c1c7af05e52fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:37.815991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c42be9e163453f871c1c7af05e52fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:37.816630 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c42be9e163453f871c1c7af05e52fa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:37.835615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2772] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:37 2022] DELETE /identity/v3/users/23c42be9e163453f871c1c7af05e52fa => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:37.840985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:37.841520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:37.842311 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:37.866797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:37.868196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:37.949016 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2773] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:37.960962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:37.963161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:38.055027 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a382-b86d-4e3b-b22d-f72167097999 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:38.059643 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:38.060297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:38.060589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:38.061022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:38.074813 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:38.076638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:38.091593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2774] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:38.738245 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:38.740137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:38.780420 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea901891-bec1-4cb6-a687-e938ac4e9621 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:38.782846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:38.783638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:38.783979 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:38.784493 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] PATH_INFO: `/v3/projects/14684a16eed9487483230deca11dd99b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:38.786694 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=14684a16eed9487483230deca11dd99b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:38.787827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:38.858871 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14684a16eed9487483230deca11dd99b', 'request_id': 'req-9edb9f37-96fa-4af2-9537-10111c6fbcfc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:38.866596 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14684a16eed9487483230deca11dd99b', 'request_id': 'req-9edb9f37-96fa-4af2-9537-10111c6fbcfc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:38.882695 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14684a16eed9487483230deca11dd99b', 'request_id': 'req-9edb9f37-96fa-4af2-9537-10111c6fbcfc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:38.903256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] The token cache is being invalidate because project 14684a16eed9487483230deca11dd99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:38.903699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:38.904715 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9edb9f37-96fa-4af2-9537-10111c6fbcfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:38.908048 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2775] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:38 2022] DELETE /identity/v3/projects/14684a16eed9487483230deca11dd99b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:38.911993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:38.928230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f07a0-0db8-4d23-bedd-414d1432100a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:38.970219 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:39.034075 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:39.034745 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.035026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.035335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] PATH_INFO: `/v3/users/eda751290dc34329933eea96cd08632d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.036619 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda751290dc34329933eea96cd08632d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:39.038703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:39.112818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda751290dc34329933eea96cd08632d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.113648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda751290dc34329933eea96cd08632d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.127786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda751290dc34329933eea96cd08632d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.156948 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda751290dc34329933eea96cd08632d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.157747 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda751290dc34329933eea96cd08632d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.172849 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2776] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:38 2022] DELETE /identity/v3/users/eda751290dc34329933eea96cd08632d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:39.232664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.233280 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.233612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.286605 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] MFA Rules not processed for user `909eaedf75674f88b41cbdc0eac33964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:39.288741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.324915 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:39.586388 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.588125 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.598131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2777] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:39.600673 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.612805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.685002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1729a-b498-49f8-9d7f-009316a5da64 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:39.691459 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:39.692111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.692425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.692699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] PATH_INFO: `/v3/projects/9c4356340add4d1abc552d3798fc52c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.694099 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4356340add4d1abc552d3798fc52c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:39.696748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:39.713391 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca15ef-7596-4f79-8bc2-78eae63e625f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:39.719590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:39.720293 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.720653 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.721129 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.722333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:39.725538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:39.763750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4356340add4d1abc552d3798fc52c4', 'request_id': 'req-61bd41d8-231f-4cfc-9c43-8c28b8607928'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.770314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:39.772051 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4356340add4d1abc552d3798fc52c4', 'request_id': 'req-61bd41d8-231f-4cfc-9c43-8c28b8607928'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.775584 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2778] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:39 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:39.779865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.781319 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4356340add4d1abc552d3798fc52c4', 'request_id': 'req-61bd41d8-231f-4cfc-9c43-8c28b8607928'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.804884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] The token cache is being invalidate because project 9c4356340add4d1abc552d3798fc52c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:39.805354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.806084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61bd41d8-231f-4cfc-9c43-8c28b8607928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:39.812974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2779] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:39 2022] DELETE /identity/v3/projects/9c4356340add4d1abc552d3798fc52c4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:39.816394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac849b-01d9-4149-8442-4176b00c3666 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.816755 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac849b-01d9-4149-8442-4176b00c3666 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.817214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac849b-01d9-4149-8442-4176b00c3666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.841627 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98ac849b-01d9-4149-8442-4176b00c3666 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:39.843006 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98ac849b-01d9-4149-8442-4176b00c3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.879590 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ba2-e823-4795-8892-d40f4f2c0ebb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:39.885840 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:39.886601 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.886878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.887146 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.887789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:39.889777 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:39.919084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2780] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:39 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:39.923755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.927270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.931824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2781] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:39.941443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.941795 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.942172 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.950008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8bf7d-1f2b-43b4-ac4c-511ca47cd851 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:39.955708 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:39.956374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:39.956651 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:39.956918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:39.957569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:39.959402 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:39.971643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2782] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:39.973289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:39.974501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:39.976109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.983529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2783] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:39.987301 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:39.993693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f7f4-2134-4c44-9ff6-1bde9c43fd50 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.003259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.003939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.004379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.004864 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] PATH_INFO: `/v3/projects/e989954ddb6348058cb5a8549a58ef03/users/c1dd245c7d05474993422e91344493e9/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.006092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b764cc2-72bb-459d-84bf-47365cbae43f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.013584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.014493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.014493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.014779 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.015404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:40.015878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.017876 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.023240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e989954ddb6348058cb5a8549a58ef03, user_id=c1dd245c7d05474993422e91344493e9, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.027999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.036472 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2784] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:26:40.038928 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.044022 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:40.051620 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf52e-4574-4c31-ba8a-069c37b611f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.055389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2785] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:39 2022] PUT /identity/v3/projects/e989954ddb6348058cb5a8549a58ef03/users/c1dd245c7d05474993422e91344493e9/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:40.057306 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.131778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.132492 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.132829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.133156 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.134507 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.137004 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.157718 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d51bc-dc02-4892-ae4f-8f9b33eac96e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.171057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.172727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.173224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.173729 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.174643 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.174781 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2786] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:40 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:40.177482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.179020 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.194949 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2787] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:40.196726 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.210283 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9955f3-b3b4-4fae-b282-4bf3f6579155 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.267085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:40.284164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.285059 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.285337 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.285664 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] PATH_INFO: `/v3/projects/e989954ddb6348058cb5a8549a58ef03/users/c1dd245c7d05474993422e91344493e9/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.288216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e989954ddb6348058cb5a8549a58ef03, user_id=c1dd245c7d05474993422e91344493e9, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.292655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.309898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2788] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:40 2022] PUT /identity/v3/projects/e989954ddb6348058cb5a8549a58ef03/users/c1dd245c7d05474993422e91344493e9/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:40.317663 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9010a-39dd-45d7-8a75-cf53d46793c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.485084 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.486853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.487481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.487923 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.488806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.505113 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.545142 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2789] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:40 2022] POST /identity/v3/users => generated 291 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:40.548782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.566258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce9468-be31-4124-bee2-469f32231e0f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.601049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.603096 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.605559 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.606627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.607600 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.615263 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.636529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2790] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:40.640423 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.657956 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03250b83-e7b2-4dee-80c4-3f692af4bb27 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.664215 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.664866 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.665140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.665426 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.667302 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.668939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834200d4-3016-408a-80b1-e49e1eb3e125 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.680717 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2791] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:40.685222 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:40.695981 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd0cd3-a751-462d-8428-18178461dbf1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:40.702021 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:40.703049 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.703606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.704024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] PATH_INFO: `/v3/projects/86aa573cf1fd400cbb44dbd3e1cf73a3/users/6643c1d3e0ca4d2eb0642cfac0f192ad/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.721191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=86aa573cf1fd400cbb44dbd3e1cf73a3, user_id=6643c1d3e0ca4d2eb0642cfac0f192ad, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:40.724087 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:40.746869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2792] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:40 2022] PUT /identity/v3/projects/86aa573cf1fd400cbb44dbd3e1cf73a3/users/6643c1d3e0ca4d2eb0642cfac0f192ad/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:40.758993 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:40.759605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:40.760026 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:40.800674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] MFA Rules not processed for user `6643c1d3e0ca4d2eb0642cfac0f192ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:40.802038 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:40.899794 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2793] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:40.902190 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.004984 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e77db-2242-4df9-9412-600303bc5eae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.005957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:41.026771 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.027299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.027585 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.027855 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.028927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.031591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.073686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2794] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:40 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:41.078372 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.096996 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d0c7f-2150-454f-aa70-ab6b218231e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.175662 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.176379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.176576 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.179169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.179169 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.184101 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.201990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:41.219083 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2795] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:41 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:41.222972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.237696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d0982-b010-45fe-86ba-4879d957c8cd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.243223 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.246206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.246904 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.247330 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.248374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.250434 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.269189 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2796] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:41.269408 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.282886 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77f42f-cdd1-471a-936f-d7f31e86477a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.292081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.293939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.294417 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.294863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] PATH_INFO: `/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d/users/27f8327dc7a3429f9c689bf328fa0ed0/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.321895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5ca4baf04248c99ef8d25420f13c1d, user_id=27f8327dc7a3429f9c689bf328fa0ed0, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.326003 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.353854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2797] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:41 2022] PUT /identity/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d/users/27f8327dc7a3429f9c689bf328fa0ed0/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:41.359122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.374513 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f2db-08f0-40ab-b208-a63281f2ef51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.532869 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.533498 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.533773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.534040 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.534698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.536353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f630f21-2362-4097-bdee-84a377f49d1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.567954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2798] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:41.572404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.585229 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15ec1-9dcd-447a-82f8-7a4bee76b374 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.594095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.594788 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.595063 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.595335 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] PATH_INFO: `/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d/users/27f8327dc7a3429f9c689bf328fa0ed0/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.597740 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b5ca4baf04248c99ef8d25420f13c1d, user_id=27f8327dc7a3429f9c689bf328fa0ed0, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.600491 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.621289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2799] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:41 2022] PUT /identity/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d/users/27f8327dc7a3429f9c689bf328fa0ed0/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:41.632685 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.633181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.633181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.680113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] MFA Rules not processed for user `27f8327dc7a3429f9c689bf328fa0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:41.681472 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:41.814800 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2800] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:41.823661 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:41.825467 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:41.905982 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122b43f-adcd-4a1c-a747-d3f94559418a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:41.909949 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:41.910573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:41.916517 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:41.917189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:41.933624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:41.935339 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:41.947542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2801] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:42.073074 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:42.093421 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f6604-ff8f-444e-a61f-7a32eb46bd03 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:42.099041 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:42.099671 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:42.099951 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:42.100834 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:42.194658 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:42.208480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:42.210241 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380697c8-ff72-4e36-8749-5a7dab632e26 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:42.227842 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:42.774683 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:42.902765 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c3ba-0be5-4752-846e-ddaa8faf4d10 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:42.910042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:42.910713 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:42.910992 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:42.911261 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] PATH_INFO: `/v3/users/6643c1d3e0ca4d2eb0642cfac0f192ad` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:42.914223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6643c1d3e0ca4d2eb0642cfac0f192ad)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:42.918132 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:43.001213 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6643c1d3e0ca4d2eb0642cfac0f192ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.001730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6643c1d3e0ca4d2eb0642cfac0f192ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.012258 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6643c1d3e0ca4d2eb0642cfac0f192ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.022569 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:43.031558 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6643c1d3e0ca4d2eb0642cfac0f192ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.032457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6643c1d3e0ca4d2eb0642cfac0f192ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.048022 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2803] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:42 2022] DELETE /identity/v3/users/6643c1d3e0ca4d2eb0642cfac0f192ad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:43.052530 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:43.052809 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:43.053076 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:43.074120 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:43.075315 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:43.141262 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2804] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:43.151448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:43.152990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:43.211536 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c3448-e46c-4e15-8bde-9f52145535d0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:43.215343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:43.215954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:43.216292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:43.216579 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:43.225950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:43.227353 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:43.237297 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2805] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:43.544140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:43.545934 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:43.562957 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbacf6-cb2f-41ca-92da-fd14edc16aa0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:43.568846 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:43.569756 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:43.570258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:43.570431 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] PATH_INFO: `/v3/projects/86aa573cf1fd400cbb44dbd3e1cf73a3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:43.571736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86aa573cf1fd400cbb44dbd3e1cf73a3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:43.589492 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:43.649417 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86aa573cf1fd400cbb44dbd3e1cf73a3', 'request_id': 'req-9d5d2a30-4731-49ee-bbe9-4689d74172a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.655900 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86aa573cf1fd400cbb44dbd3e1cf73a3', 'request_id': 'req-9d5d2a30-4731-49ee-bbe9-4689d74172a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.666766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86aa573cf1fd400cbb44dbd3e1cf73a3', 'request_id': 'req-9d5d2a30-4731-49ee-bbe9-4689d74172a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.683887 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] The token cache is being invalidate because project 86aa573cf1fd400cbb44dbd3e1cf73a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:43.684286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.685112 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.686939 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2806] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:43 2022] DELETE /identity/v3/projects/86aa573cf1fd400cbb44dbd3e1cf73a3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:43.690767 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:43.717909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b450-5cd7-42ae-8825-70c174891c56 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:43.806130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:43.807585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:43.808359 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:43.808987 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] PATH_INFO: `/v3/users/27f8327dc7a3429f9c689bf328fa0ed0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:43.812161 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f8327dc7a3429f9c689bf328fa0ed0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:43.814660 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:43.905811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f8327dc7a3429f9c689bf328fa0ed0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.906572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f8327dc7a3429f9c689bf328fa0ed0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.927001 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f8327dc7a3429f9c689bf328fa0ed0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.949175 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f8327dc7a3429f9c689bf328fa0ed0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.949677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-56ee1a85-8ef6-4d08-aa83-900c45fde063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f8327dc7a3429f9c689bf328fa0ed0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:43.982854 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2807] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:43 2022] DELETE /identity/v3/users/27f8327dc7a3429f9c689bf328fa0ed0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:44.432308 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:44.433744 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:44.453362 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d2a30-4731-49ee-bbe9-4689d74172a8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:44.546418 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:44.547259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:44.547534 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:44.548162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] PATH_INFO: `/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:44.549412 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b5ca4baf04248c99ef8d25420f13c1d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:44.551762 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:44.571240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:44.615119 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b5ca4baf04248c99ef8d25420f13c1d', 'request_id': 'req-d88ad5d5-9620-40b6-aaca-fc330c13613c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:44.627690 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b5ca4baf04248c99ef8d25420f13c1d', 'request_id': 'req-d88ad5d5-9620-40b6-aaca-fc330c13613c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:44.642196 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b5ca4baf04248c99ef8d25420f13c1d', 'request_id': 'req-d88ad5d5-9620-40b6-aaca-fc330c13613c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:44.667894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] The token cache is being invalidate because project 3b5ca4baf04248c99ef8d25420f13c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:44.668611 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:44.669527 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d88ad5d5-9620-40b6-aaca-fc330c13613c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:44.676696 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2808] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:44 2022] DELETE /identity/v3/projects/3b5ca4baf04248c99ef8d25420f13c1d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:44.678649 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153e19-8fed-410e-af72-155034005e86 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:44.679114 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153e19-8fed-410e-af72-155034005e86 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:44.679584 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153e19-8fed-410e-af72-155034005e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:44.688858 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:44.708887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5c153e19-8fed-410e-af72-155034005e86 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:44.713028 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c153e19-8fed-410e-af72-155034005e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:44.808477 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2809] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:44.817252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:44.817660 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:44.818188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:44.849908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:44.851353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:44.862710 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2810] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:44.867194 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c153e19-8fed-410e-af72-155034005e86 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:44.878709 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c153e19-8fed-410e-af72-155034005e86 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:44.888501 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:44.889265 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:44.889543 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:44.889814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:44.890622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:44.890926 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:44.892694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:44.909847 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2811] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 11:26:44.911364 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:44.923622 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb0b94-820c-4578-ae30-7ae7eef9d63b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:44.934551 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:44.935657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:44.936154 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:44.937216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:44.938095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:44.941098 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:44.980374 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2812] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:44 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:44.981457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:44.997783 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c6d41-1a73-46d9-9517-5905d7bd6e5a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.081766 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.083456 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.083888 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.084244 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.084906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.087547 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.125243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2813] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:44 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:45.130162 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.145890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f4c7d-d41b-420e-b0a4-0779b9ee40ea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.153072 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.153798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.154108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.154372 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.155017 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.156805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.167766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2814] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:45.172002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.183585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a1967-203f-48ff-940e-44560588af46 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.188870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.189636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.189977 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.190314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.191042 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.192843 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f32466-0c13-4093-888b-ce1002ca5eaa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.208511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2815] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:45.210103 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.222847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835a41b-f0c7-4071-b4b0-cc191184193c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.227516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.228178 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.228545 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.228811 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] PATH_INFO: `/v3/projects/1ebd73119be84bbdb3ff1046d118d7ff/users/4ec566e054e14092ae6cf567a7c402cb/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.245531 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebd73119be84bbdb3ff1046d118d7ff, user_id=4ec566e054e14092ae6cf567a7c402cb, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.248161 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.265646 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2816] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:45 2022] PUT /identity/v3/projects/1ebd73119be84bbdb3ff1046d118d7ff/users/4ec566e054e14092ae6cf567a7c402cb/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:45.276731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.277076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.277371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.315498 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] MFA Rules not processed for user `4ec566e054e14092ae6cf567a7c402cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:45.317122 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:45.415040 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:45.424480 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2817] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:45.434975 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.436793 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:45.508334 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24ca82-8855-46a0-b550-35edc2c81dbd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.512986 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.513607 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.513884 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.514151 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.524202 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.525804 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14f1f7-cb2f-4dfa-9825-4f0477cfbf55 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.540212 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2818] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:45.560532 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.645686 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356e132-da78-4417-901f-75fc2d92f709 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:45.653352 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:45.654014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.654318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.654590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] PATH_INFO: `/v3/users/4ec566e054e14092ae6cf567a7c402cb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.655939 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec566e054e14092ae6cf567a7c402cb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:45.657933 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:45.737635 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec566e054e14092ae6cf567a7c402cb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:45.739281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec566e054e14092ae6cf567a7c402cb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:45.753691 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec566e054e14092ae6cf567a7c402cb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:45.781086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec566e054e14092ae6cf567a7c402cb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:45.782805 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec566e054e14092ae6cf567a7c402cb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:45.801292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2819] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:45 2022] DELETE /identity/v3/users/4ec566e054e14092ae6cf567a7c402cb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:45.806249 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c292f-e80c-412d-96e3-fca57a42cf5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.806529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c292f-e80c-412d-96e3-fca57a42cf5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.807067 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c292f-e80c-412d-96e3-fca57a42cf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:45.844175 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-657c292f-e80c-412d-96e3-fca57a42cf5d None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:45.845438 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-657c292f-e80c-412d-96e3-fca57a42cf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:45.898457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:45.924008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2820] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:45.935523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:45.937164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:45.977057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:45.977546 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:45.977972 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.008009 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f707bf6-0005-4a4b-a2e3-c81487792a03 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.012820 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.013523 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.013863 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.014189 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.026376 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.027904 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.032982 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] MFA Rules not processed for user `c1dd245c7d05474993422e91344493e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:46.034675 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.039800 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2821] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:46.138128 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2822] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:46.147400 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.149871 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.186375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:46.261738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90788f-2c64-4cc0-a34d-5777269a3525 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.265829 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32cbfe0d7b584729b45b85c65377648f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.266438 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.266711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.266975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.269057 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.330624 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.332096 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb81dc-63aa-4fa1-b4c8-fca99d344704 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.341563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2823] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:46.375221 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.376768 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.391109 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b413ff-d481-4339-9c4c-4bfd3bd5ec27 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.396882 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.397627 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.397954 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.398299 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] PATH_INFO: `/v3/projects/1ebd73119be84bbdb3ff1046d118d7ff` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.399918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebd73119be84bbdb3ff1046d118d7ff)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.402110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.482566 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebd73119be84bbdb3ff1046d118d7ff', 'request_id': 'req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:46.488518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebd73119be84bbdb3ff1046d118d7ff', 'request_id': 'req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:46.497945 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723bd3e-dc32-4c68-9805-668bb9850d6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.498308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723bd3e-dc32-4c68-9805-668bb9850d6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.498542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723bd3e-dc32-4c68-9805-668bb9850d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.500908 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebd73119be84bbdb3ff1046d118d7ff', 'request_id': 'req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:46.514323 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] The token cache is being invalidate because project 1ebd73119be84bbdb3ff1046d118d7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:46.514714 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:46.515451 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4953fc7b-20f7-44d9-ae63-acfba95d1a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:46.518150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2824] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:46 2022] DELETE /identity/v3/projects/1ebd73119be84bbdb3ff1046d118d7ff => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:46.523097 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.523480 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.524142 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.540846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f723bd3e-dc32-4c68-9805-668bb9850d6d None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:46.542402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f723bd3e-dc32-4c68-9805-668bb9850d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.548246 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:46.549540 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.603514 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:46.615784 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2825] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:46.630056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.630468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.630738 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.632256 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2826] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:46.655846 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:46.657230 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.666925 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2827] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:46.671264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.677942 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.679447 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:46.686353 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333ad8-6ffc-43b5-b4bf-bc541d49d9b3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.691642 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.692300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.692591 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.692862 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.693515 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:46.693818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.695457 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.708595 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2828] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:26:46.712569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.723445 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e55df-076d-43a4-88de-f5921f712a81 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.728258 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.729122 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.729404 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.729670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.730301 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.732216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.742884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:46.758256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28d20-9634-4aea-b7f6-997d85713550 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.762755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2829] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:46 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:26:46.765317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.766272 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.766574 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.766848 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.766950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.776815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.778402 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.786692 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2830] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:26:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 11:26:46.862254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab32cbd-8788-438c-a8bc-f27f50f55c88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.868081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.868741 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.869158 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.869525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.870166 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.872095 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.902377 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2831] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:46 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:26:46.903585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.914924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ee2d6-1010-456d-83ca-436a162c8c8a service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.924689 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.925312 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.925590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.925862 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.926511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.931622 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.943605 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2832] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:46.948487 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.959635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017c002-937d-4095-93b6-3f9f5eb92e70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:46.965883 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:46.966496 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:46.966770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:46.967032 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:46.967684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:46.969346 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cc303-7f8c-4c70-a352-405de85e5098 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:46.980970 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2833] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:46.985409 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:46.998092 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d854708-5ae1-426f-8cc9-b0965582f11a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.004757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.005688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.006123 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.006549 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] PATH_INFO: `/v3/projects/ec0e303e7aeb45148c36e8def34af6ce/users/dc0d2061c2094214b76a5ac244625726/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.025184 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0e303e7aeb45148c36e8def34af6ce, user_id=dc0d2061c2094214b76a5ac244625726, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.028137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.049095 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2834] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:46 2022] PUT /identity/v3/projects/ec0e303e7aeb45148c36e8def34af6ce/users/dc0d2061c2094214b76a5ac244625726/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:47.060938 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.061266 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.061539 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.093063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] MFA Rules not processed for user `dc0d2061c2094214b76a5ac244625726`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:47.094295 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:47.173757 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2835] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:47.178572 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.274777 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1e0a8-02dc-4963-bd0c-dccfc1baa138 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.280055 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.280932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.281217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.281658 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.282292 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.284468 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.325909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2836] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:47.327030 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.338231 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f4c2-0e08-4977-8908-4dcaf6cf491a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.363051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:47.409177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.410029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.412117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.412117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.412117 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.413528 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.441670 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2837] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 11:26:47.446054 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.457947 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c74f9e-273a-4861-9a3c-b372c969f477 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.467435 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.469008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.469427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.469902 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.470481 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.472367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.486720 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2838] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:47.491509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.502569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53946ea8-6cd6-4f23-b646-f7a7cec32345 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.507170 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.507797 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.508088 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.508359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.509110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.510801 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c701980-9708-4bcf-b0a9-412465c85c00 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.526378 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2839] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:47.528069 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.539749 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede942dd-a89d-4186-93bc-b23051602f9c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.544520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.545180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.545455 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.545726 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] PATH_INFO: `/v3/projects/050c82337f744affb2b98a3531cfd501/users/937c3f3232db4fd3a2fd339bb1e6f69b/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.561396 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=050c82337f744affb2b98a3531cfd501, user_id=937c3f3232db4fd3a2fd339bb1e6f69b, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.564445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.579898 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2840] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:47 2022] PUT /identity/v3/projects/050c82337f744affb2b98a3531cfd501/users/937c3f3232db4fd3a2fd339bb1e6f69b/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:47.590827 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.591525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.592081 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.633423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] MFA Rules not processed for user `937c3f3232db4fd3a2fd339bb1e6f69b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:47.636639 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:47.721358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2841] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:47.725648 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.801183 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097102-7b83-4bb7-bb00-a7b22572b12f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.807682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.808550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.808950 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.809354 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.810091 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.812097 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.818531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:47.850221 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2842] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:47.851727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.865815 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed7e7-f9b6-4543-930d-28e34da5d4a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:47.943425 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:47.944058 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:47.944357 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:47.944630 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:47.945489 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:47.947526 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:47.975650 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2843] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:47 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:47.979750 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:47.995811 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb710498-a517-4afb-9a3c-df87095cd35d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:48.000748 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:48.001366 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.001654 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.001915 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.002567 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:48.004164 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:48.019853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2844] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:48.021124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:48.035399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b5af6-cd9a-4148-a019-586e9e69a070 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:48.040363 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:48.041191 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.041772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.042140 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] PATH_INFO: `/v3/projects/e6d2e30f15b7419fa095e3e653df2471/users/bfc5aec3eace488f853b5d548d5d4ee2/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.061012 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d2e30f15b7419fa095e3e653df2471, user_id=bfc5aec3eace488f853b5d548d5d4ee2, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:48.063772 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:48.076521 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:48.087818 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2845] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:48 2022] PUT /identity/v3/projects/e6d2e30f15b7419fa095e3e653df2471/users/bfc5aec3eace488f853b5d548d5d4ee2/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:48.091995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:48.107300 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb535ba-719d-471c-9e9b-07c6ae6da6f4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:48.177137 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:48.177778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.178047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.178311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.178958 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:48.180563 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bc9e1-62cf-49ed-8bd2-c2cf2ded907d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:48.192870 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2846] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:48.196894 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:48.210682 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c55a8-f6b2-4bc6-9c4b-250baad1603c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:48.215525 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:48.216195 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.216518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.216789 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] PATH_INFO: `/v3/projects/e6d2e30f15b7419fa095e3e653df2471/users/bfc5aec3eace488f853b5d548d5d4ee2/roles/46b0a9dd588a4810a6ec59469cb69284` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.218716 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d2e30f15b7419fa095e3e653df2471, user_id=bfc5aec3eace488f853b5d548d5d4ee2, role_id=46b0a9dd588a4810a6ec59469cb69284)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:48.221500 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:48.232783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2847] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:48 2022] PUT /identity/v3/projects/e6d2e30f15b7419fa095e3e653df2471/users/bfc5aec3eace488f853b5d548d5d4ee2/roles/46b0a9dd588a4810a6ec59469cb69284 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:48.304086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.304086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.304086 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.341531 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] MFA Rules not processed for user `bfc5aec3eace488f853b5d548d5d4ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:48.342753 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:48.443371 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2848] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:48.454569 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:48.456628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:48.577554 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578b4e0-f525-426e-b755-43971d66e178 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:48.586432 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:48.587281 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:48.587807 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:48.588240 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:48.685121 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:48.686906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:48.697816 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2849] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:26:50.392031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:50.394031 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:50.408445 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b897b5-b3bf-4d0d-8e25-955e502a52a5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:50.412772 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:50.413841 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:50.414337 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:50.414694 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:50.503274 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:50.507959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:50.510911 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:50.532681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:51.342150 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:51.351699 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac6b2d-42f8-4adb-81bb-e64386ba83c1 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:51.356054 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:51.357010 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:51.357286 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:51.357556 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:51.370918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:51.372541 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9cb08-6824-4426-a9d7-31af734f1fe3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:51.383733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2851] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:54.859594 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:54.861151 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:54.946894 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582515a-417e-451d-b7c3-dc0ab52dc553 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:54.951990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:54.952723 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:54.953078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:54.953404 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] PATH_INFO: `/v3/users/34efeb60abd8417ba53962a45fd3e5a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:54.954601 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34efeb60abd8417ba53962a45fd3e5a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:54.956655 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:55.010427 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34efeb60abd8417ba53962a45fd3e5a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.011377 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34efeb60abd8417ba53962a45fd3e5a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.023144 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34efeb60abd8417ba53962a45fd3e5a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.038580 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34efeb60abd8417ba53962a45fd3e5a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.039379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34efeb60abd8417ba53962a45fd3e5a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.055281 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2852] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:54 2022] DELETE /identity/v3/users/34efeb60abd8417ba53962a45fd3e5a0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:55.060393 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.062300 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.063439 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.087669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:55.089282 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.167180 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2853] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:55.177443 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:55.178998 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.262778 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07dc506-f68a-4b73-a19a-da7cae13a859 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:55.266860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:55.267503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.267776 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.268044 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.279573 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:55.281214 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:55.293475 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2854] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:55.577482 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:55.579228 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.594767 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f67f3-2394-44be-9520-82158fcc40d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:55.602055 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:55.602925 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.603224 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.603524 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] PATH_INFO: `/v3/users/dc0d2061c2094214b76a5ac244625726` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.605165 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0d2061c2094214b76a5ac244625726)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:55.609046 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:55.641518 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:55.643089 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:55.644682 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.669666 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c148403-794a-4b8c-a294-9960b42e5322 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:55.690824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:55.692989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.693965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.695120 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] PATH_INFO: `/v3/projects/1f51871afe1c49f4a472b09e633ad1b9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.696110 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0d2061c2094214b76a5ac244625726'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.697466 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f51871afe1c49f4a472b09e633ad1b9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:55.708986 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:55.720879 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0d2061c2094214b76a5ac244625726'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.731094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:55.732795 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0d2061c2094214b76a5ac244625726'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.734127 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0d2061c2094214b76a5ac244625726'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.762683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3582864-aa6c-47a5-9674-88cec03acc8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0d2061c2094214b76a5ac244625726'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.772398 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2855] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:55 2022] DELETE /identity/v3/users/dc0d2061c2094214b76a5ac244625726 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:55.776847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.776847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.777264 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.791638 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f51871afe1c49f4a472b09e633ad1b9', 'request_id': 'req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.798385 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f51871afe1c49f4a472b09e633ad1b9', 'request_id': 'req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.802024 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:55.803405 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.805449 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f51871afe1c49f4a472b09e633ad1b9', 'request_id': 'req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.825204 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] The token cache is being invalidate because project 1f51871afe1c49f4a472b09e633ad1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:55.825653 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.826704 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3d1bffe5-acbf-46db-b5f2-cef56387ae4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:55.828924 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2856] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:55 2022] DELETE /identity/v3/projects/1f51871afe1c49f4a472b09e633ad1b9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:55.833554 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ec0a-5d4d-4ab1-aaaf-607727d2fead None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.834615 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ec0a-5d4d-4ab1-aaaf-607727d2fead None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.834895 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ec0a-5d4d-4ab1-aaaf-607727d2fead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.863550 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9986ec0a-5d4d-4ab1-aaaf-607727d2fead None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:55.864817 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9986ec0a-5d4d-4ab1-aaaf-607727d2fead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.883962 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2857] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:55.892409 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:55.893965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.947786 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2858] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:55.956326 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.957035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.957308 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.976734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb718d3-cc94-46b6-8a7a-d8bacf685f83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:55.980240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:55.980889 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:55.981497 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:55.981638 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:55.981912 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:55.982251 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:55.990527 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1445/2859] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:55.992399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:55.994312 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:55.996012 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.004898 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2860] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 11:26:56.005399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71ee21-d290-458a-bf7e-a2b279acb285 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.011981 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.012698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.013317 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.013479 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.014185 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 11:26:56.014539 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.016177 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.033109 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1446/2861] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 9 11:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 11:26:56.038147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.049896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b29330-628c-4267-8904-59f84fdb71a7 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.055043 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.055890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.056333 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.056806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.057614 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.059508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e8e06-7c21-4f95-8a62-8cba5fc2430f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.100220 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2862] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 9 11:26:56 2022] POST /identity/v3/projects => generated 397 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:56.102287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.112990 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0dc1c-c86a-426c-8109-1f1d46ad0136 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.150131 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63585-f248-465d-8fd7-848cab89087b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.150389 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63585-f248-465d-8fd7-848cab89087b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.150565 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce63585-f248-465d-8fd7-848cab89087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.172612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fce63585-f248-465d-8fd7-848cab89087b None None] MFA Rules not processed for user `909eaedf75674f88b41cbdc0eac33964`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:56.173895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fce63585-f248-465d-8fd7-848cab89087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:56.183008 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.183965 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.184448 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.184932 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.185959 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.186878 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:56.190641 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.225241 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1447/2863] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 9 11:26:56 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 11:26:56.229879 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.239815 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5663b5-075a-4c8c-8611-85d940e1d6fc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.241689 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2864] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:56.248232 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.251600 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.251991 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.252488 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.253273 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.255280 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.262582 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:56.269806 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1448/2865] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 11:26:56.273538 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce63585-f248-465d-8fd7-848cab89087b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.288491 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce63585-f248-465d-8fd7-848cab89087b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.293253 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.293976 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.294344 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.294701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.295502 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.297618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.310941 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2866] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 9 11:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 11:26:56.313585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.326667 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132fca7-f043-44b3-a4a4-769a3f7a9da2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.337770 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.338395 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.339429 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.339620 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:56.340153 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.340764 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] PATH_INFO: `/v3/projects/d356026db8e74eeb933595598a0d0b23/users/357d90cd14964e0b89fb4fc2eb20cfe9/roles/3a91cfd137cf4404b2ed6d52380e69d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.355739 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcac576-8515-4651-a228-396a3149410e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.360773 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.361516 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.361891 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.362510 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] PATH_INFO: `/v3/projects/ec0e303e7aeb45148c36e8def34af6ce` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.363861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0e303e7aeb45148c36e8def34af6ce)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.366275 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.374195 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d356026db8e74eeb933595598a0d0b23, user_id=357d90cd14964e0b89fb4fc2eb20cfe9, role_id=3a91cfd137cf4404b2ed6d52380e69d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.377052 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581d6be-a6f2-4181-a93d-84549aa7b70f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.396037 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1449/2867] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 9 11:26:56 2022] PUT /identity/v3/projects/d356026db8e74eeb933595598a0d0b23/users/357d90cd14964e0b89fb4fc2eb20cfe9/roles/3a91cfd137cf4404b2ed6d52380e69d5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:56.408071 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.408368 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.408636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.431552 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0e303e7aeb45148c36e8def34af6ce', 'request_id': 'req-d40df2d6-fc46-4127-a088-c9bc433db85f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.437873 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] MFA Rules not processed for user `357d90cd14964e0b89fb4fc2eb20cfe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:56.438501 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0e303e7aeb45148c36e8def34af6ce', 'request_id': 'req-d40df2d6-fc46-4127-a088-c9bc433db85f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.439000 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:56.449594 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0e303e7aeb45148c36e8def34af6ce', 'request_id': 'req-d40df2d6-fc46-4127-a088-c9bc433db85f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.463676 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] The token cache is being invalidate because project ec0e303e7aeb45148c36e8def34af6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:56.464066 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.465270 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.467099 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2868] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:56 2022] DELETE /identity/v3/projects/ec0e303e7aeb45148c36e8def34af6ce => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:56.470886 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.472367 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:56.509708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1450/2869] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 11:26:56.522349 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:56.524442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:56.547992 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40df2d6-fc46-4127-a088-c9bc433db85f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.552635 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.553259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.553529 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.553792 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] PATH_INFO: `/v3/users/937c3f3232db4fd3a2fd339bb1e6f69b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.554906 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937c3f3232db4fd3a2fd339bb1e6f69b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.556896 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.590711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d059d06-eebc-4600-85ce-ff2520c23a9f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:56.594818 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:56.595434 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:56.595708 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:56.595969 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:56.606267 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:56.607757 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:56.617143 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1451/2870] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:56.621078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:56.632755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937c3f3232db4fd3a2fd339bb1e6f69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.644918 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937c3f3232db4fd3a2fd339bb1e6f69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.658388 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937c3f3232db4fd3a2fd339bb1e6f69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.659077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937c3f3232db4fd3a2fd339bb1e6f69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.674083 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937c3f3232db4fd3a2fd339bb1e6f69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:56.681259 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2871] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:56 2022] DELETE /identity/v3/users/937c3f3232db4fd3a2fd339bb1e6f69b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:26:57.098056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:57.116859 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cebf0-a10f-4521-9bfe-e81279b54bc8 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:57.230899 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:57.231551 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:57.231824 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:57.232094 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] PATH_INFO: `/v3/projects/050c82337f744affb2b98a3531cfd501` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:57.233515 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] RBAC: Authorizing `identity:delete_project(project_id=050c82337f744affb2b98a3531cfd501)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:57.236070 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:57.250688 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:57.309018 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050c82337f744affb2b98a3531cfd501', 'request_id': 'req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.322320 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050c82337f744affb2b98a3531cfd501', 'request_id': 'req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.329906 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050c82337f744affb2b98a3531cfd501', 'request_id': 'req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.346240 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] The token cache is being invalidate because project 050c82337f744affb2b98a3531cfd501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:57.346690 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.347444 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.349085 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1452/2872] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:57 2022] DELETE /identity/v3/projects/050c82337f744affb2b98a3531cfd501 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:57.353416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:57.355460 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:57.372895 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a3bbe-6c2d-49e3-801a-591e41800c57 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:57.451499 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:57.452211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:57.453254 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:57.453588 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] PATH_INFO: `/v3/users/bfc5aec3eace488f853b5d548d5d4ee2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:57.454946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc5aec3eace488f853b5d548d5d4ee2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:57.457359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:57.524453 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc5aec3eace488f853b5d548d5d4ee2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.531965 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc5aec3eace488f853b5d548d5d4ee2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.544736 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc5aec3eace488f853b5d548d5d4ee2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.545669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc5aec3eace488f853b5d548d5d4ee2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.552687 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:57.554522 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:57.560310 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc5aec3eace488f853b5d548d5d4ee2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:57.567406 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2873] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:26:57 2022] DELETE /identity/v3/users/bfc5aec3eace488f853b5d548d5d4ee2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:57.659606 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa3b0c-fa6a-4b5f-8b3c-22033162b20e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:57.727356 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:57.728552 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:57.728731 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:57.728979 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:57.730909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:57.743268 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:57.746397 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603c7ee-6061-4d3e-9a24-c50d0ab9eb0e service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:57.759015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1453/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:26:57.927084 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:57.928752 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:57.990446 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:58.023273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633168e-0b27-41b1-8186-09c1db60a5bb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:58.029504 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:58.030077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:58.030345 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:58.030783 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] PATH_INFO: `/v3/projects/e6d2e30f15b7419fa095e3e653df2471` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:58.032107 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d2e30f15b7419fa095e3e653df2471)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:58.034063 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:58.084985 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d2e30f15b7419fa095e3e653df2471', 'request_id': 'req-105707df-bf19-4069-b8e6-20d61dd33eef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:58.091875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d2e30f15b7419fa095e3e653df2471', 'request_id': 'req-105707df-bf19-4069-b8e6-20d61dd33eef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:58.100943 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d2e30f15b7419fa095e3e653df2471', 'request_id': 'req-105707df-bf19-4069-b8e6-20d61dd33eef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:58.113791 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] The token cache is being invalidate because project e6d2e30f15b7419fa095e3e653df2471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:26:58.114239 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:58.115057 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:26:58.116654 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2875] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:26:57 2022] DELETE /identity/v3/projects/e6d2e30f15b7419fa095e3e653df2471 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:26:58.682389 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:58.682988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:58.682988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:58.704867 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:26:58.706107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:58.712248 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:26:58.770589 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1454/2876] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 9 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:26:58.784335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:26:58.785890 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:26:58.865748 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105707df-bf19-4069-b8e6-20d61dd33eef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:26:58.871786 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:26:58.872426 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:26:58.872711 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:26:58.872978 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:26:58.882628 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:26:58.884252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1162471-6c8c-4eb8-9e96-80e78b80edb6 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:26:58.893216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2877] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:26:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:27:04.017048 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:04.022393 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:04.093294 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17eecd-5580-4491-b0c8-2913e7872e2b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:04.099343 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:04.100073 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:04.100442 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:04.100790 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] PATH_INFO: `/v3/users/357d90cd14964e0b89fb4fc2eb20cfe9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:04.102076 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] RBAC: Authorizing `identity:delete_user(user_id=357d90cd14964e0b89fb4fc2eb20cfe9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:04.104826 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:04.164174 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357d90cd14964e0b89fb4fc2eb20cfe9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.164937 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357d90cd14964e0b89fb4fc2eb20cfe9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.188559 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357d90cd14964e0b89fb4fc2eb20cfe9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.203321 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357d90cd14964e0b89fb4fc2eb20cfe9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.204425 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357d90cd14964e0b89fb4fc2eb20cfe9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.221540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1455/2878] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:27:04 2022] DELETE /identity/v3/users/357d90cd14964e0b89fb4fc2eb20cfe9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:27:04.226181 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:04.226684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:04.226796 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:04.247188 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] MFA Rules not processed for user `9ebf3927bad74de89873a93b84ee2eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:27:04.248450 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:04.312743 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:27:04.326321 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2879] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:27:04.336962 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:04.339328 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:04.387373 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:27:04.409839 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6fc06-237e-4df6-a8a7-a7df621d340c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:04.413698 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:04.414107 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:04.414379 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:04.414642 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:04.424975 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:04.426256 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:04.437590 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1456/2880] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:27:04.743575 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:04.757419 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:04.772865 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707668e6-c78f-4854-9564-405c4b5cfa1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:04.777924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:04.778684 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:04.779399 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:04.779848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] PATH_INFO: `/v3/projects/d356026db8e74eeb933595598a0d0b23` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:04.781192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d356026db8e74eeb933595598a0d0b23)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:04.783829 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:04.847447 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd356026db8e74eeb933595598a0d0b23', 'request_id': 'req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.853769 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd356026db8e74eeb933595598a0d0b23', 'request_id': 'req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.866474 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd356026db8e74eeb933595598a0d0b23', 'request_id': 'req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.879733 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] The token cache is being invalidate because project d356026db8e74eeb933595598a0d0b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:27:04.880019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.880693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:27:04.882252 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2881] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:27:04 2022] DELETE /identity/v3/projects/d356026db8e74eeb933595598a0d0b23 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:27:14.433223 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:27:14.877025 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:27:25.207155 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:25.208940 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:25.221453 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d255f47-6b9a-42de-9ab1-97f587512229 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:25.272664 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:25.273252 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:25.273520 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:25.273780 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:25.276047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:25.339104 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:25.340611 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:25.352754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1457/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:27:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:27:35.346415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=50 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:27:51.610657 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:51.612177 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:51.625669 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3a56-73d3-46c6-8e1e-ab7082cfd43c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:51.629508 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:51.630108 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:51.630373 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:51.631077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:51.632927 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:51.723430 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:51.724924 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:51.745727 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2883] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:27:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:27:52.951130 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:27:52.960636 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b463f-f65d-44cb-9c2e-f62d26df3696 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:27:52.965983 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:27:52.966394 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:27:52.966669 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:27:52.966936 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:27:52.969014 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:27:52.984093 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:27:52.985542 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2681ca9-044b-4f65-b40c-f689406240f8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:27:52.997243 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1458/2884] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:27:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:01.743210 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=43 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:02.991860 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:09.483379 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:09.485375 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:09.550359 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef0091-e725-48e3-9ef0-990b7842bd6f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:09.557261 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'befb4776ff7b4014997d32e4697f29a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:09.558184 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:09.558616 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:09.559029 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:09.562380 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:09.630116 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:09.631550 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:09.640861 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2885] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:28:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:16.586782 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:16.587330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:16.587330 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:16.607742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] MFA Rules not processed for user `5b4c4030b5fb4381a598a54588880b75`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:28:16.608988 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:16.616699 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1459/2886] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 9 11:28:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:28:19.227766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:19.229386 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:19.242271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e4134-ca86-4c24-8ac2-4dbefb7a386b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:19.246192 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:19.246806 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:19.247078 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:19.247340 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:19.249208 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:19.260229 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:19.261950 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:19.270990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2887] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:28:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:26.602119 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:29.267020 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=54 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:29.821509 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:29.823277 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:29.840216 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89b056-7af5-4a2f-836a-397d2c0a9cb1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:29.845927 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:29.846533 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:29.846807 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:29.847080 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:29.848869 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:29.861503 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:29.863074 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:29.873907 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1460/2888] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:28:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:33.553147 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:33.567799 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823da6e-cc16-41a1-9020-db8133feda55 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:33.574641 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:33.575416 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:33.575734 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:33.575990 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:33.578164 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:33.592219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:33.593832 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda6b8e-9bbe-4f86-b0e5-816bb564d975 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:33.604701 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2889] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:28:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:39.869631 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:43.600219 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:48.980493 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:48.982367 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:48.996995 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0c79e-8519-40dd-97b5-5814246c618f service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:49.004140 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbbd789ff1f244eb98d2e5dd001a81c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:49.004900 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:49.005171 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:49.005470 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:49.007291 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:49.021355 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:49.022954 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:49.034555 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1461/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:28:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:28:50.242621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:50.243155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:50.243155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:50.294199 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] MFA Rules not processed for user `baa77a1e4ec94a159eccbee7db74bcf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 11:28:50.295390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:50.360853 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2891] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 9 11:28:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 11:28:50.402483 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:50.404035 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:50.500254 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3cd42-4731-4c91-b9b0-de1ce709c946 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:50.505625 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f55820319ba442aa8288cf179681b2d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:50.506289 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:50.506610 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:50.506822 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:50.515078 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:50.516673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:50.524887 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1462/2892] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 9 11:28:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 11:28:57.173874 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:57.176755 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:57.244124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a14937-2c16-4032-ae47-37020d79f0cb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:57.248693 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:57.249335 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:57.249606 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:57.249875 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] PATH_INFO: `/v3/users/909eaedf75674f88b41cbdc0eac33964` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:57.251004 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] RBAC: Authorizing `identity:delete_user(user_id=909eaedf75674f88b41cbdc0eac33964)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:57.252963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:57.300509 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '909eaedf75674f88b41cbdc0eac33964'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.307794 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '909eaedf75674f88b41cbdc0eac33964'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.315999 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '909eaedf75674f88b41cbdc0eac33964'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.316655 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '909eaedf75674f88b41cbdc0eac33964'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.324782 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:57.330248 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '909eaedf75674f88b41cbdc0eac33964'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.337211 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2893] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:28:57 2022] DELETE /identity/v3/users/909eaedf75674f88b41cbdc0eac33964 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:28:57.523529 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:57.535677 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadc7fe-93e7-4be6-8b69-e02acc14b3bb service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:57.595264 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:57.595891 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:57.596163 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:57.596593 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] PATH_INFO: `/v3/projects/2b3b0a56e9724ccd81df765dfc49739d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:57.597754 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3b0a56e9724ccd81df765dfc49739d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:57.600117 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:57.648540 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3b0a56e9724ccd81df765dfc49739d', 'request_id': 'req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.654375 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3b0a56e9724ccd81df765dfc49739d', 'request_id': 'req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.655433 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:28:57.662883 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3b0a56e9724ccd81df765dfc49739d', 'request_id': 'req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.678568 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] The token cache is being invalidate because project 2b3b0a56e9724ccd81df765dfc49739d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:28:57.678974 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.679742 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.681383 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1463/2894] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:28:57 2022] DELETE /identity/v3/projects/2b3b0a56e9724ccd81df765dfc49739d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:28:57.685621 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:57.687137 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:57.701271 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1432e-cf0b-43ae-8f2f-39982bda28de admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:57.766495 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:57.767124 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:57.767390 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:57.767659 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] PATH_INFO: `/v3/users/c1dd245c7d05474993422e91344493e9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:57.768847 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1dd245c7d05474993422e91344493e9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:57.770805 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:57.826437 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1dd245c7d05474993422e91344493e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.834008 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1dd245c7d05474993422e91344493e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.843798 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1dd245c7d05474993422e91344493e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.844612 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1dd245c7d05474993422e91344493e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.858205 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1dd245c7d05474993422e91344493e9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:57.864618 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2895] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:28:57 2022] DELETE /identity/v3/users/c1dd245c7d05474993422e91344493e9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:28:58.053081 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:28:58.054711 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:28:58.067701 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98457f49-fbf4-4795-ba0d-d2d423b0c4cd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:28:58.130361 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:28:58.131029 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:28:58.131314 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:28:58.131674 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] PATH_INFO: `/v3/projects/e989954ddb6348058cb5a8549a58ef03` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:28:58.132964 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e989954ddb6348058cb5a8549a58ef03)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:28:58.134844 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:28:58.185705 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e989954ddb6348058cb5a8549a58ef03', 'request_id': 'req-79d7d711-cc65-4a8c-883c-0df2956add07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:58.191217 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e989954ddb6348058cb5a8549a58ef03', 'request_id': 'req-79d7d711-cc65-4a8c-883c-0df2956add07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:58.197420 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e989954ddb6348058cb5a8549a58ef03', 'request_id': 'req-79d7d711-cc65-4a8c-883c-0df2956add07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:58.212771 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] The token cache is being invalidate because project e989954ddb6348058cb5a8549a58ef03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:28:58.213056 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:58.213703 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:28:58.215173 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1464/2896] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:28:58 2022] DELETE /identity/v3/projects/e989954ddb6348058cb5a8549a58ef03 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:29:07.861051 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=56,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:29:08.208989 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=55,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:30:18.374145 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:18.375765 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:18.435867 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924683f7-98bd-41b2-a5fb-5a2eb051f3ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:18.484931 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3216e5d1c354d8e8d5c1fbf449f882c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:18.485573 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:18.485848 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:18.486115 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:18.488216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:18.560737 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:18.562183 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:18.572216 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2897] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:30:24.964246 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:24.965853 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:25.022872 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:25.024719 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:25.045963 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af539b9-db95-4163-ad51-f282fc970d35 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:25.053411 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:25.054625 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:25.055019 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:25.055423 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] PATH_INFO: `/v3/users/5b4c4030b5fb4381a598a54588880b75` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:25.056760 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4c4030b5fb4381a598a54588880b75)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:25.058813 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:25.059399 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d711-cc65-4a8c-883c-0df2956add07 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:25.065062 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:25.065673 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:25.065943 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:25.066206 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:25.067909 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:25.079696 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:25.081309 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:25.092287 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1465/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:30:25.112991 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4c4030b5fb4381a598a54588880b75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.120520 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4c4030b5fb4381a598a54588880b75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.124522 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:30:25.131517 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4c4030b5fb4381a598a54588880b75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.132787 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4c4030b5fb4381a598a54588880b75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.145942 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4c4030b5fb4381a598a54588880b75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.152318 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2899] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:30:25 2022] DELETE /identity/v3/users/5b4c4030b5fb4381a598a54588880b75 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:30:25.342832 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:25.354603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f02a5-13cd-401c-8290-c9cace2b8b0d service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:25.417415 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:25.418046 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:25.418318 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:25.418585 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] PATH_INFO: `/v3/projects/bfd671fcff7e4b24b5415a3a1b70ad0f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:25.419730 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd671fcff7e4b24b5415a3a1b70ad0f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:25.421679 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:25.469833 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd671fcff7e4b24b5415a3a1b70ad0f', 'request_id': 'req-60c36a90-f944-4587-b2d2-20387ba96d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.475930 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd671fcff7e4b24b5415a3a1b70ad0f', 'request_id': 'req-60c36a90-f944-4587-b2d2-20387ba96d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.483002 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd671fcff7e4b24b5415a3a1b70ad0f', 'request_id': 'req-60c36a90-f944-4587-b2d2-20387ba96d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.499884 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] The token cache is being invalidate because project bfd671fcff7e4b24b5415a3a1b70ad0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:30:25.500179 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.500816 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.502311 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1466/2900] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:30:25 2022] DELETE /identity/v3/projects/bfd671fcff7e4b24b5415a3a1b70ad0f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:30:25.506953 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:25.509354 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:25.523273 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1124e-397c-4806-baa5-eb094cd9ba78 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:25.584238 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:25.584808 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:25.585062 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:25.585358 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] PATH_INFO: `/v3/users/f404b89b457643c381d6afaed16e8c3c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:25.586722 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f404b89b457643c381d6afaed16e8c3c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:25.588686 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:25.640113 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f404b89b457643c381d6afaed16e8c3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.647077 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f404b89b457643c381d6afaed16e8c3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.656068 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f404b89b457643c381d6afaed16e8c3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.656713 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f404b89b457643c381d6afaed16e8c3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.668928 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f404b89b457643c381d6afaed16e8c3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:25.675572 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2901] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 9 11:30:25 2022] DELETE /identity/v3/users/f404b89b457643c381d6afaed16e8c3c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 11:30:25.872111 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:30:25.873566 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:30:25.888665 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c36a90-f944-4587-b2d2-20387ba96d72 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:30:25.908511 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:30:25.953546 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ebf3927bad74de89873a93b84ee2eae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69344eba9b82497c957a2d6fd891ec79', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:30:25.954180 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:30:25.954457 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:30:25.954724 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] PATH_INFO: `/v3/projects/59b2dd90807744e0a327315f3725605e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:30:25.955878 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b2dd90807744e0a327315f3725605e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:30:25.957941 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:30:26.008796 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b2dd90807744e0a327315f3725605e', 'request_id': 'req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:26.014603 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b2dd90807744e0a327315f3725605e', 'request_id': 'req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:26.020814 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b2dd90807744e0a327315f3725605e', 'request_id': 'req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:26.036015 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] The token cache is being invalidate because project 59b2dd90807744e0a327315f3725605e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 11:30:26.036286 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:26.037047 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ddd7ce0c-7257-46b4-8b02-2d9e452ea6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 11:30:26.038681 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1467/2902] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 9 11:30:25 2022] DELETE /identity/v3/projects/59b2dd90807744e0a327315f3725605e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 11:30:35.672187 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:30:36.033238 dv-cinder-852514-2-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 11:32:05.743155 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 11:32:05.745206 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:32:05.827698 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2948f08-ec2e-4afa-b145-eaa8db200807 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 11:32:05.890819 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9792491cfe3c489a83639e77a27bde04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6b6327b5cd60427eb93d32c22ec52860', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 11:32:05.891414 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 11:32:05.891683 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 11:32:05.891946 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 11:32:05.893766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 11:32:05.904969 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 11:32:05.906549 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02699b19-3edc-4d75-9a79-64e857153362 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 11:32:05.917603 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2903] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 9 11:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 11:32:15.912766 dv-cinder-852514-2-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}